Re: Breakage in httpd 2.4.39/windows/cmake?

2019-07-15 Thread Stefan Eissing
The source modules/http2/h2_ngn_shed.c was removed in 2.4.39, but it was still 
in the CMakeList. In branch 2.4.x it was fixed.

We need a build server that uses cmake, it seems. Anyone an idea how to set 
that up?

> Am 12.07.2019 um 22:58 schrieb Steve Hay :
> 
> On Fri, 12 Jul 2019 at 21:41, William A Rowe Jr  wrote:
>> 
>> Because of some trouble on another project, I wanted to recheck
>> the current nghttp2 build between our mutual dependencies and
>> httpd... and something isn't looking so healthy in the CMake
>> build of the last release. (But I did answer my question, bazel
>> shouldn't be failing to build nghttp2, it is rock hard stable in terms
>> of building rev 1.39.1) CMake is rev 3.9.4 in this example of our
>> build below.
>> 
> 
> I did a build of 2.4.39 a few days ago using "cmake version
> 3.14.19050301-MSVC_2" and everything was fine.



Re: mod_md v2.0.x + mod_ssl backport

2019-07-12 Thread Stefan Eissing



> Am 12.07.2019 um 09:53 schrieb Joe Orton :
> 
> On Wed, Jul 10, 2019 at 01:40:10PM +0200, Stefan Eissing wrote:
>> Added descriptions for this. 
>> 
>> Updated the backport patch. Updated the mod_md version in trunk, 2.4.x 
>> and github master and github maintenance branch. At this time, each 
>> change is about half a day's work.
> 
> Thanks, all LGTM!

Thanks!

>> Ceterum censeo, it's a pain to develop something for 2.4.x. And I 
>> forgot the reason why this is so...
> 
> We do at least try not to break the stable release all the time by 
> having RTC for 2.4.x, I am not sure what is so unusual here.

Maybe it's just me then. That we stayed backwards compatible for a decade(?) 
now has advantages for our users, but it comes at a cost. For me personally, it 
is doubling the work in integration and testing.

Cheers, Stefan

> Regards, Joe



Re: mod_md v2.0.x + mod_ssl backport

2019-07-10 Thread Stefan Eissing
One day later...

> Am 09.07.2019 um 13:28 schrieb Joe Orton :
> 
> On Tue, Jul 09, 2019 at 11:57:00AM +0200, Stefan Eissing wrote:
>> mod_md v2.0.x has landed in 2.4.x. This offers the ACMEv2 (RFC 8555) support 
>> and offers various monitoring possibilities for admins to see what is going 
>> on. But...it really needs votes for a mod_ssl related backport:
>> 
>>  mod_ssl: offer new hooks for modules like mod_md and remove use of mod_md's 
>> optional functions.
>> 
>> Without this, the new TLS challenge will not work and people still need to 
>> rely on port 80. I heard from one user whose ISP is blocking incoming port 
>> 80 (everything can be reasoned with "security" nowadays). So, for some, this 
>> is really vital.
>> 
>> I'd appreciate if some would find the time for a vote.
> 
> Sorry, should have looked when you asked before...

Thanks for looking at it, nevertheless.

> The new hooks should be in mod_ssl_openssl.h and use OpenSSL types 
> properly for type-safety rather than void *, that's exactly the kind of 
> thing this header was added for.  Or else it should marshall everything 
> through DER, not sure if that is feasible, probably less efficient.

Ok, learned something new.

> Not sure why 'struct apr_array_header_t' was used but I committed the 
> trivial fix for that.

Just not a big fan of nested includes myself. But it's not that important.

> The API is a bit less documented that I'd like though I know mod_ssl is 
> worse than awful here, e.g. a few places not clear what the return value 
> means, I have to guess/look at the implementation, this is ignored:
> 
> +ssl_run_add_cert_files(s, p, pks->cert_files, pks->key_files);
> 
> but this one is not:
> 
>if (ssl_run_get_stapling_status(, , conn, s, x) == 
> APR_SUCCESS) {
> 
> and when passing apr_array_header_t make clear it's an array of (const 
> char *) because again otherwise you have to guess/read the code

Added descriptions for this. 

Updated the backport patch. Updated the mod_md version in trunk, 2.4.x and 
github master and github maintenance branch. At this time, each change is about 
half a day's work. 

Cheers, Stefan

Ceterum censeo, it's a pain to develop something for 2.4.x. And I forgot the 
reason why this is so...

Re: mod_md v2.0.x + mod_ssl backport

2019-07-09 Thread Stefan Eissing



> Am 09.07.2019 um 13:28 schrieb Joe Orton :
> 
> On Tue, Jul 09, 2019 at 11:57:00AM +0200, Stefan Eissing wrote:
>> mod_md v2.0.x has landed in 2.4.x. This offers the ACMEv2 (RFC 8555) support 
>> and offers various monitoring possibilities for admins to see what is going 
>> on. But...it really needs votes for a mod_ssl related backport:
>> 
>>  mod_ssl: offer new hooks for modules like mod_md and remove use of mod_md's 
>> optional functions.
>> 
>> Without this, the new TLS challenge will not work and people still need to 
>> rely on port 80. I heard from one user whose ISP is blocking incoming port 
>> 80 (everything can be reasoned with "security" nowadays). So, for some, this 
>> is really vital.
>> 
>> I'd appreciate if some would find the time for a vote.
> 
> Sorry, should have looked when you asked before...

I am even more sorry that the person originally requesting this change never 
replied back.

This all makes me very sad. 

> The new hooks should be in mod_ssl_openssl.h and use OpenSSL types 
> properly for type-safety rather than void *, that's exactly the kind of 
> thing this header was added for.  Or else it should marshall everything 
> through DER, not sure if that is feasible, probably less efficient.
> 
> Not sure why 'struct apr_array_header_t' was used but I committed the 
> trivial fix for that.
> 
> The API is a bit less documented that I'd like though I know mod_ssl is 
> worse than awful here, e.g. a few places not clear what the return value 
> means, I have to guess/look at the implementation, this is ignored:
> 
> +ssl_run_add_cert_files(s, p, pks->cert_files, pks->key_files);
> 
> but this one is not:
> 
>if (ssl_run_get_stapling_status(, , conn, s, x) == 
> APR_SUCCESS) {
> 
> and when passing apr_array_header_t make clear it's an array of (const 
> char *) because again otherwise you have to guess/read the code.

*shrugs*

mod_md v2.0.x + mod_ssl backport

2019-07-09 Thread Stefan Eissing
mod_md v2.0.x has landed in 2.4.x. This offers the ACMEv2 (RFC 8555) support 
and offers various monitoring possibilities for admins to see what is going on. 
But...it really needs votes for a mod_ssl related backport:
  
  mod_ssl: offer new hooks for modules like mod_md and remove use of mod_md's 
optional functions.

Without this, the new TLS challenge will not work and people still need to rely 
on port 80. I heard from one user whose ISP is blocking incoming port 80 
(everything can be reasoned with "security" nowadays). So, for some, this is 
really vital.

I'd appreciate if some would find the time for a vote.

Cheers, Stefan

Re: [RFC] fixing mod_cgid error logging

2019-07-05 Thread Stefan Eissing
No objections from me, as I lack the expertise here. Patch looks readable, 
though. ;)

> Am 05.07.2019 um 12:57 schrieb Joe Orton :
> 
> PR 54211 and 60692 track a design problem in mod_cgid: the stderr of 
> spawned CGI scripts is a copy of the main server's stderr.  This is a 
> significant regression from mod_cgi (you lose logging prefixes, 
> per-vhost config, non-file/pipe-logging provider support e.g. syslog)
> 
> I can think of two main approaches to fix this:
> 
> 1) enhance the client/server protocol which cgid uses to be a bit more 
> like FastCGI with headers, record types, etc and multiplex both stderr 
> and stdout over the Unix socket.  We'd need a new thread or process for 
> each new spawned script child to translate CGI stdout/stderr into this 
> protocol, or to completely redesign the cgid_server main loop to be 
> event-driven.  Plus a new bucket type similar to the CGI bucket which 
> handles the protocol client side.
> 
> 2) Create a new pipe for stderr in the client and pass this to the 
> server using Unix fd passing magic for the CGI script to use as stderr.  
> Factor out the CGI bucket from mod_cgi and use this as-is in mod_cgid.
> 
> I think (1) might be a cleaner design in the long run but (2) is going 
> to be vastly simpler to implement.  (2) might take some effort from 
> platform maintainers to work across all Unixes.
> 
> I'm going to run with (2) in two steps unless there are major 
> objections.
> 
> a) make fd passing work (opt-in via configure switch) if ErrorLog is an 
> fd, by passing that fd directly to the server (fixing PR 60692 only) - 
> PoC attached to show this is relatively simple, +100 lines
> 
> b) then make that work via a new pipe with the CGI bucket abstracted out 
> and used across both mod_cgi/cgid (properly fixing PR 54211)
> 
> Regards, Joe
> 
> 



Re: connection state handling and KeepAliveTimeout

2019-07-03 Thread Stefan Eissing
The change works nicely in my tests. I have some where I trigger timeout and 
keepalive and I see the effect.

For 2.4.x I test as well and need a slight adjustment to event.c. Could you 
look if that ok? Then I would propose that for backport.

Cheers, Stefan



h2-keepalive-yann-v2.patch
Description: Binary data

> Am 03.07.2019 um 14:08 schrieb Yann Ylavic :
> 
> On Wed, Jul 3, 2019 at 2:03 PM Stefan Eissing
>  wrote:
>> 
>>> Am 03.07.2019 um 13:57 schrieb Yann Ylavic :
>>> 
>>> On Wed, Jul 3, 2019 at 1:50 PM Stefan Eissing
>>>  wrote:
>>>> 
>>>> HE IS ALIVE! \o/
>>> 
>>> Yes :)) still a bit overwhelmed at day $job though :/
>> 
>> It's a shabby world where open source devs need to pay rent and food+drinks 
>> in restaurants...
> 
> You're telling me!



Re: connection state handling and KeepAliveTimeout

2019-07-03 Thread Stefan Eissing



> Am 03.07.2019 um 13:57 schrieb Yann Ylavic :
> 
> On Wed, Jul 3, 2019 at 1:50 PM Stefan Eissing
>  wrote:
>> 
>> HE IS ALIVE! \o/
> 
> Yes :)) still a bit overwhelmed at day $job though :/

It's a shabby world where open source devs need to pay rent and food+drinks in 
restaurants...

Re: connection state handling and KeepAliveTimeout

2019-07-03 Thread Stefan Eissing
HE IS ALIVE! \o/

Great! I will try this in trunk later today. That there are two "queues", one 
with Timeout and one with KeepAliveTimeout, fixed for all connections, was my 
read, but I am not that deep into mpm_event. Thanks for confirming and the 
patch.

Will report back.

Cheers, Stefan

> Am 03.07.2019 um 13:44 schrieb Yann Ylavic :
> 
> On Wed, Jul 3, 2019 at 9:23 AM Stefan Eissing
>  wrote:
>> 
>>> Am 02.07.2019 um 14:00 schrieb Eric Covener :
>>> 
>>> I think a successful exit of CONN_STATE_WRITE_COMPLETION would
>>> conceptually lead to keepalive state so that would be plausible.  And
>>> a failed one would go into lingering close.
>> 
>> That is also how I understand it.
> 
> I think WRITE_COMPLETION, as his name does _not_ indicate, can also
> handle _read_ completion by using/setting CONN_SENSE_WANT_READ.
> 
>> With the current mpm architecture, there are only two situations where
>> connection processing by h2 returns to the mpm:
>> 1. when all streams(request) have been handled and the connection is really 
>> in KeepAlive
>> 2. when the flow-control windows of existing streams are exhausted. The only 
>> thing that can unlock this state are new frames from the client.
>> 
>> My understanding is that the mpm either times out the connection or receives 
>> client data which it then makes invoke connection processing again. Here, 
>> the KeepAliveTimeout seems to alway apply which is wrong for the situation 2 
>> described above.
> 
> So you'd want Timeout, right?
> 
>>> An option in this area is trunk-only PT_USER callback in event which
>>> frees you from shoehorning into the states and just lets you get
>>> called back when a socket is readable.
>>> This is what's used in mod_proxy_wstunnell in the opt-in async mode.
>> 
>> Hmm, I need to look at that. Thanks for the pointer.
> 
> That (but trunk only and quite huge change for 2.4.x), or as said
> above CONN_SENSE_WANT_READ.
> However the MPM event's queues (write_completion_q, keepalive_q, ...)
> handle _fixed_ timeouts only (using fast/dumb APR_RINGs in O(1)), so
> CONN_SENSE_WANT_READ could work only if h2 needs Timeout instead of
> KeepAliveTimeout.
> If a "dynamic" timeout is needed, PT_USER is the only option I think.
> 
> So possibly something along the lines of the attached patch would be
> enough, modulo CONN_SENSE_WANT_READ which you know better where/how to
> set ;)
> 
> 
> Regards,
> Yann.
> 




Re: connection state handling and KeepAliveTimeout

2019-07-03 Thread Stefan Eissing



> Am 02.07.2019 um 14:00 schrieb Eric Covener :
> 
> On Tue, Jul 2, 2019 at 7:38 AM Stefan Eissing
>  wrote:
>> 
>> 
>> 
>>> Am 02.07.2019 um 13:35 schrieb Eric Covener :
>>> 
>>> On Tue, Jul 2, 2019 at 4:51 AM Stefan Eissing
>>>  wrote:
>>>> 
>>>> People with in insight into your connection state handling, please review:
>>>> 
>>>> https://bz.apache.org/bugzilla/show_bug.cgi?id=63534
>>>> 
>>>> describes an issue with HTTP/2 and KeepAliveTimeout. Basically, HTTP/2 
>>>> writes response DATA, gets blocked on HTTP/2 flow control,
>>>> and makes a blocking READ on the connection. Since the only thing that can 
>>>> move things forward is a packet from the client.
>>>> 
>>>> This triggers the KeepAlive behaviour of our connections, which is not 
>>>> what should apply here. The alternatives I can see are:
>>>> 1. non-blocking reads and therefore continue to block a worker
>>>> 2. twiddle the server->keepalive setting, question is, will that influence 
>>>> the mpm?
>>> 
>>> Is it actually keepalive behavior/processing or just the servers
>>> keepalive timeout explicitly set before the outstanding read on the
>>> master connection?
>>> 
>>> I don't exactly follow how, but it seems like w/ event mod_h2 already
>>> knows how to suspend/resume based on CONN_STATE_WRITE_COMPLETION when
>>> the session thread knows it's idle too long.
>>> But maybe there is an issue in how the connection handler returns and
>>> event goes into lingering close instead?
>> 
>> What I think is happening is that connection state is 
>> CONN_STATE_WRITE_COMPLETION and h2 does a BLOCKing read on the main 
>> connection. After the write has been completed, the keepalive handling 
>> applies to the connection - and pre-close()/linger()/closes it.
>> 
>> But the HTTP/2 state is not suitable for keepalive, just timeout. Is there a 
>> way to avoid that?
> 

Thanks for taking the time to discuss this. :)

> I think a successful exit of CONN_STATE_WRITE_COMPLETION would
> conceptually lead to keepalive state so that would be plausible.  And
> a failed one would go into lingering close.

That is also how I understand it.

> This touches back to where I didn't really understand how h2 gets
> control back after explicitly yielding via
> CONN_STATE_WRITE_COMPLETION.  Do you know if h2 currently/ever hops
> mpm threads this way successfully?

With the current mpm architecture, there are only two situations where
connection processing by h2 returns to the mpm:
1. when all streams(request) have been handled and the connection is really in 
KeepAlive
2. when the flow-control windows of existing streams are exhausted. The only 
thing that can unlock this state are new frames from the client.

My understanding is that the mpm either times out the connection or receives 
client data which it then makes invoke connection processing again. Here, the 
KeepAliveTimeout seems to alway apply which is wrong for the situation 2 
described above.

> Another option is the pre-existing timed callabck
> ap_mpm_register_timed_callback() once you get into this stall where
> you really want to give a thread back but still have hope it might
> progress.

The problem is not a stall, but that mpm shuts down the connection too early. 
Or I misunderstood your point...

> An option in this area is trunk-only PT_USER callback in event which
> frees you from shoehorning into the states and just lets you get
> called back when a socket is readable.
> This is what's used in mod_proxy_wstunnell in the opt-in async mode.

Hmm, I need to look at that. Thanks for the pointer.

Cheers, Stefan

> -- 
> Eric Covener
> cove...@gmail.com



Re: connection state handling and KeepAliveTimeout

2019-07-02 Thread Stefan Eissing



> Am 02.07.2019 um 13:35 schrieb Eric Covener :
> 
> On Tue, Jul 2, 2019 at 4:51 AM Stefan Eissing
>  wrote:
>> 
>> People with in insight into your connection state handling, please review:
>> 
>> https://bz.apache.org/bugzilla/show_bug.cgi?id=63534
>> 
>> describes an issue with HTTP/2 and KeepAliveTimeout. Basically, HTTP/2 
>> writes response DATA, gets blocked on HTTP/2 flow control,
>> and makes a blocking READ on the connection. Since the only thing that can 
>> move things forward is a packet from the client.
>> 
>> This triggers the KeepAlive behaviour of our connections, which is not what 
>> should apply here. The alternatives I can see are:
>> 1. non-blocking reads and therefore continue to block a worker
>> 2. twiddle the server->keepalive setting, question is, will that influence 
>> the mpm?
> 
> Is it actually keepalive behavior/processing or just the servers
> keepalive timeout explicitly set before the outstanding read on the
> master connection?
> 
> I don't exactly follow how, but it seems like w/ event mod_h2 already
> knows how to suspend/resume based on CONN_STATE_WRITE_COMPLETION when
> the session thread knows it's idle too long.
> But maybe there is an issue in how the connection handler returns and
> event goes into lingering close instead?

What I think is happening is that connection state is 
CONN_STATE_WRITE_COMPLETION and h2 does a BLOCKing read on the main connection. 
After the write has been completed, the keepalive handling applies to the 
connection - and pre-close()/linger()/closes it.

But the HTTP/2 state is not suitable for keepalive, just timeout. Is there a 
way to avoid that?

> 
> (sorry if I am totally misunderstanding)



connection state handling and KeepAliveTimeout

2019-07-02 Thread Stefan Eissing
People with in insight into your connection state handling, please review:

https://bz.apache.org/bugzilla/show_bug.cgi?id=63534

describes an issue with HTTP/2 and KeepAliveTimeout. Basically, HTTP/2 writes 
response DATA, gets blocked on HTTP/2 flow control,
and makes a blocking READ on the connection. Since the only thing that can move 
things forward is a packet from the client.

This triggers the KeepAlive behaviour of our connections, which is not what 
should apply here. The alternatives I can see are:
1. non-blocking reads and therefore continue to block a worker
2. twiddle the server->keepalive setting, question is, will that influence the 
mpm?

Thanks for your help in this,

Stefan

Re: mod_mdv2: stapling

2019-06-26 Thread Stefan Eissing
Please have a look in trunk if the current implenentation is what you had in 
mind.

Cheers, Stefan

> Am 24.06.2019 um 17:39 schrieb Graham Leggett :
> 
> On 24 Jun 2019, at 17:25, Stefan Eissing  wrote:
> 
>> You mean optional hooks by mod_ssl so that mod_md or someone else can take 
>> over?
> 
> Yes.
> 
> I while back I was looking at supporting an arbitrary collections of 
> certificates instead of discrete certs per virtual hosts, and the md optional 
> function was right where a hook would go. I've been meaning to fix this, but 
> I’m drowing in stuff right now.
> 
> Regards,
> Graham
> —
> 



cmake help wanted

2019-06-25 Thread Stefan Eissing
I added the cmake definitions for mod_md in r1862041. If someone could find the 
time to run this on Windows in the coming weeks, that'd be great.

- Stefan

Re: svn commit: r1841225 - /httpd/httpd/trunk/modules/dav/main/props.c

2019-06-25 Thread Stefan Eissing
Nice work!

> Am 25.06.2019 um 10:44 schrieb Joe Orton :
> 
> 



Re: mod_mdv2: stapling

2019-06-24 Thread Stefan Eissing
You mean optional hooks by mod_ssl so that mod_md or someone else can take over?

> Am 24.06.2019 um 17:23 schrieb Graham Leggett :
> 
> On 24 Jun 2019, at 17:12, Stefan Eissing  wrote:
> 
>> General interworking mod_ssl <-> mod_md: 2 new optional functions:
> 
> One quick thing I wanted to bring up a while back - rather than optional 
> functions which can only ever be provided by a single implementation, can 
> these be hooks instead?
> 
> A hook allows additional modules to modify the behaviour if we want to in the 
> future, without replacing mod_md.
> 
> Regards,
> Graham
> —
> 



mod_mdv2: stapling

2019-06-24 Thread Stefan Eissing
I am looking for feedback and harsh critics from this excellent group of people 
here.
If you see mistakes or have ideas on improving, I'd appreciate it.

Cheers,

Stefan


The new OCSP stapling implementation in mod_md will:

- be for server certificates in virtual hosts
- co-exist with existing mod_ssl ocsp stapling
- be watchdog driven, file system persisted

Features of mod_ssl stapling I do not plan to implement:

- SSLStaplingFakeTryLater: 
  either we have a response or not. if not, nothing is set in the response. On 
must-staple, clients will fail.
- SSLStaplingForceURL: 
  think I do not need it for the test setup.
- SSLStaplingResponseMaxAge: 
  there will be a "renew window" instead. So watchdog will get a new response x 
amount of time before the existing expires
- SSLStaplingResponseTimeSkew: 
  I see no need.
- SSLStaplingReturnResponderErrors: 
  error from OCSP responders are detected by the watchdog and logged. Clients 
only see valid stapling or no stapling.

Maybe I am missing a use case here. If you are aware of one (e.g. need for time 
skew), please let me know.


General interworking mod_ssl <-> mod_md: 2 new optional functions:

  apr_status_t md_stapling_init_cert(server_rec *s, X509 *cert, ...)
  apr_status_t md_stapling_get_response(md_oscp_response **prsp, server_rec *s, 
X509 *cert, conn_rec *c...)

Via "MDStapling on|off", the admin can control the new stapling for all or just 
a particular MD.
Via "MDStapling all", the new stapling would apply to all certificates, even 
those not covered by an MD.

md_stapling_init_cert(...) will return:
  APR_SUCCESS, when mod_md takes over stapling of this server_rec
  APR_ENOTIMPL, when it does not and mod_ssl shall continue as it does now
  otherwise, a real error happened.

md_stapling_get_response(...) will return:
  APR_SUCCESS with a valid response
  APR_ENOENT if no valid response is available
  APR_ENOTIMPL if mod_md does not provide stapling for this server/cert
  otherwise, a real error happened.




libcrurl

2019-06-19 Thread Stefan Eissing
I just want to mention that I have no plans to adapt the upcoming Chrome 
library libcrurl (see 
https://daniel.haxx.se/blog/2019/06/19/google-to-reimplement-curl-in-libcrurl/ 
).

And, boy, do I hope they have no plans to do the same with our product names.

Cheers, Stefan

Re: configure --enable-maintainer-mode gives errors on fedora 30

2019-06-18 Thread Stefan Eissing
I always run in maintainer mode, but on MacOS. gcc is often more picky. Is that 
what you use?

> Am 18.06.2019 um 14:33 schrieb jean-frederic clere :
> 
> Hi,
> 
> I have spotted a bunch of errors when using ./configure
> --enable-maintainer-mode on the branch 2.4.x, do we plan to fix those?
> 
> Some are really errors (probably core with bad httpd,conf) some are more
> tricky...
> 
> -- 
> Cheers
> 
> Jean-Frederic



Re: ACME support in Apache web server

2019-06-14 Thread Stefan Eissing
Hi Abul,

there are no plans to add support for this in Apache. You know of CAs who 
require/use it? I have read the feature and can imagine what it might be used 
for, but so far I have not seen an implementation in the wild.

Best Regards, Stefan

> Am 13.06.2019 um 17:49 schrieb Abul Salek :
> 
> Hello, 
>  
> Is there a plan to support the “external account binding” feature of ACME 
> (RFC 8555) natively in the Apache web server?
>  
> Best regards,
>  
> Abul Salek
> Product Management, Sectigo



Re: Is it safe to stop reading a bucket brigade in an input filter before the end?

2019-05-20 Thread Stefan Eissing



> Am 19.05.2019 um 13:11 schrieb Sorin Manolache :
> 
> On 16/05/2019 00.26, Paul Callahan wrote:
>> I have an apache body filter that copies off data from the incoming
>> request.   But I terminate the reading of the data if the size of the
>> accumulated data is over some limit.
>> This is my function, it just breaks  out of the loop when the max is read.
>>  It seems to work ok.   Do I need to do any additional cleanup or anything
>> because I did not go to the end?
> 
> Hello,
> 
> I suppose your code is ok.
> 
> Keep in mind that apache must clear the request body in order to be able to 
> parse and serve a new request that arrives on same, reused TCP connection.
> 
> So towards the end of the processing of the current request, apache calls a 
> function, I think it's called ap_discard_request_body. This function will 
> read data from the network socket and pass it to the filter chain. So your 
> input filter will be called again (unless you called ap_remove_input_filter 
> somewhere), after you've already sent the response to the client.

Good advice. When your input filter aborts (it needs to return an error code), 
the connection is in an undefined state for HTTP/1.1. You could make sure that 
the connection->keepalive flag is set to AP_CONN_CLOSE. Otherwise, as Sorin 
said, the dicard_request_body() might attempt to read the rest of the input 
that hangs before the next request.

Cheers, Stefan


> Best regards,
> Sorin
> 
> 
>> Thank you
>> // returns 0 after desired length is read
>> int my_append_data(const char *data, apr_size_t len, void *request_ctx);
>> void my_body_read_done(void *request_ctx);
>> apr_status_t my_body_input_filter(ap_filter_t *f, apr_bucket_brigade *out_bb,
>>  ap_input_mode_t mode,
>> apr_read_type_e block, apr_off_t nbytes,
>> void *request_ctx) {
>> request_rec *r = f->r;
>> conn_rec *c = r->connection;
>> apr_bucket_brigade *tmp_bb;
>> int ret;
>> tmp_bb = apr_brigade_create(r->pool, c->bucket_alloc);
>> if (APR_BRIGADE_EMPTY(tmp_bb)) {
>> ret = ap_get_brigade(f->next, tmp_bb, mode, block, nbytes);
>> if (mode == AP_MODE_EATCRLF || ret != APR_SUCCESS)
>> return ret;
>> }
>> while (!APR_BRIGADE_EMPTY(tmp_bb)) {
>> apr_bucket *in_bucket = APR_BRIGADE_FIRST(tmp_bb);
>> apr_bucket *out_bucket;
>> const char *data;
>> apr_size_t len;
>> if (APR_BUCKET_IS_EOS(in_bucket)) {
>> APR_BUCKET_REMOVE(in_bucket);
>> APR_BRIGADE_INSERT_TAIL(out_bb, in_bucket);
>> my_body_read_done(request_ctx);
>> break;
>> }
>> ret = apr_bucket_read(in_bucket, , , block);
>> if (ret != APR_SUCCESS) {
>> return ret;
>> }
>> // copy read data up to a limit of 1mb, then stop.
>> if (!my_append_data(data, len, request_ctx)) {
>> apr_bucket_delete(in_bucket);
>> break;
>> }
>> out_bucket = apr_bucket_heap_create(data, len, 0, c->bucket_alloc);
>> APR_BRIGADE_INSERT_TAIL(out_bb, out_bucket);
>> apr_bucket_delete(in_bucket);
>> }
>> return APR_SUCCESS;
>> }
> 



Re: Questions with mod_proxy_http2

2019-05-14 Thread Stefan Eissing



> Am 14.05.2019 um 09:30 schrieb Ruediger Pluem :
> 
> 
> 
> On 05/13/2019 03:30 PM, Stefan Eissing wrote:
>> 
>> 
>>> Am 13.05.2019 um 14:42 schrieb Plüm, Rüdiger, Vodafone Group 
>>> :
>>> 
>>> I recently started using mod_proxy_http2 and some questions popped up for 
>>> me:
>>> 
>>> 1. Why do we retry 5 times hardcoded (leading to AH10023 in case we fail)?
>>>  The other protocols only try to retry once if the ping failed (additional 
>>> Expect 100 header in the HTTP case, PING packet in the AJP case).
>> 
>> I think that is buried in history. I cannot think of a good reason for it.
> 
> Thanks for confirmation. For trunk my immediate idea would be to change '5' 
> to '2' and thus be somewhat in line with the
> other schemas. But I guess this is not possible for 2.4.x due to 
> compatibility concerns. I guess we need to make it
> configurable there with a default of 5. Opinions?

mod_proxy_http2 is still experimental. Feel free to change anything that 
improves in your setup!

>> 
>>> 2. Could we try to leverage the ping configuration parameter for workers 
>>> used by HTTP / AJP by sending a PING frame on the HTTP/2 backend
>>>  connection and wait for the reply for the configured seconds in ping, 
>>> retry once if failed with a new connection and return 503 if failed again
>>>  or continue processing if things were fine?
>> 
>> Is that something the module can provide directly or is this for the 
>> proxy/balancer infrastructure?
> 
> This is something the module provides. It just needs to reply with a 503 if 
> the worker is seen as faulty. This causes
> a possibly configured loadbalancer that has this worker as member to fail 
> over to a different member.
> 
> I understand the modules current behavior as follows:
> 
> 1. If the TCP connection is reused and no frame was received within the last 
> second a PING frame is added before
>   the stream for the request is set up.
> 2. The request body if any is not sent until the PING reply has arrived.
> 3. If the ping does not arrive reply with 503.

Yes, that was my intention. I added it to prevent a race with a GOAWAY frame 
from the backend (e.g. its keepalive timed out).

> The issue I see with the above is:
> 
> 1. Once the request is sent only idempotent requests could be sent to another 
> worker in case we have a loadbalancer
>   setup. Once a non idempotent request is sent we cannot be sure if it was 
> not processed somehow by the backend.
> 2. Even if for new connections a TCP connection can be established it is not 
> clear if the backend is ready to process it
>   due to things like backlogs, deferred accept setups or separate accept 
> threads. Using a different short ping
>   timeout removes these uncertainties. With regards to the ping overhead this 
> should only be done if the admin
>   configured the ping and hence is aware of the additional overhead with 
> respect to traffic and latency.
> 
> A similar behavior to the other modules would be
> 
> 1. Sent the PING frame no matter if it is a new or existing connection in 
> case the ping option is configured on
>   the worker.
> 2. Wait for the time configured in the ping option for the PING reply to 
> arrive. If it does not arrive reply with a 503,
>   if it does continue with the request.

I agree to your observation in regard to idempotency. Holding back the body 
alone is not enough. Always sending a PING simplifies things, although for a 
new connection, the arrival of the remote SETTINGS frames should be indiction 
enough that the HTTP/2 is sound. But since latency to backend usually is low, 
this may not be an issue.

- Stefan

> 
> Regards
> 
> Rüdiger



Re: mod_md version 2

2019-05-14 Thread Stefan Eissing
Thanks!

> Am 14.05.2019 um 09:02 schrieb Ruediger Pluem :
> 
> 
> 
> On 05/06/2019 02:53 PM, Stefan Eissing wrote:
>> Heya,
>> 
>> the beautiful people at MOSS, Mozilla's Open Source Support, decided to give 
>> me a grant for Let's Encrypt and Stapling improvements in Apache! Big thanks!
>> 
>> I described what I plan to do here: 
>> https://github.com/icing/mod_md/wiki/V2Design
>> 
>> There are also github issues for collecting feedback and I pointed people to 
>> the Apache users mailing list as well.
>> 
>> Besides the support for ACMEv2, which is in-scope of the module, I plan to 
>> add a new OCSP stapling implementation in the module as well. That may lead 
>> to some head scratching here and I want to explain my reasoning and, 
>> ideally, get feedback from you.
> 
> Great to hear this. I digged out some discussions from the past that might be 
> useful (some even started by you :-)):
> 
> https://lists.apache.org/thread.html/1a61e9dfbd685c4102b097e8189bccb7d5da39bf9f32fcbe7407a760@%3Cdev.httpd.apache.org%3E
> 
> https://lists.apache.org/thread.html/040a5ef30dbe7649b88c24cd9716eaf4c47d2d800f4a6858508d4fab@%3Cdev.httpd.apache.org%3E
> 
> 
> Regards
> 
> Rüdiger
> 



Re: Questions with mod_proxy_http2

2019-05-13 Thread Stefan Eissing



> Am 13.05.2019 um 14:42 schrieb Plüm, Rüdiger, Vodafone Group 
> :
> 
> I recently started using mod_proxy_http2 and some questions popped up for me:
> 
> 1. Why do we retry 5 times hardcoded (leading to AH10023 in case we fail)?
>   The other protocols only try to retry once if the ping failed (additional 
> Expect 100 header in the HTTP case, PING packet in the AJP case).

I think that is buried in history. I cannot think of a good reason for it.

> 2. Could we try to leverage the ping configuration parameter for workers used 
> by HTTP / AJP by sending a PING frame on the HTTP/2 backend
>   connection and wait for the reply for the configured seconds in ping, retry 
> once if failed with a new connection and return 503 if failed again
>   or continue processing if things were fine?

Is that something the module can provide directly or is this for the 
proxy/balancer infrastructure?

> Regards
> 
> Rüdiger
> 
> C2 General

-Stefan

Re: eor bucket

2019-05-09 Thread Stefan Eissing
Thanks, Yann. Just proposed these for backport.

> Am 09.05.2019 um 12:04 schrieb Yann Ylavic :
> 
> r1707362



Re: eor bucket

2019-05-09 Thread Stefan Eissing



> Am 09.05.2019 um 11:32 schrieb Yann Ylavic :
> 
> The issue is more that the hooks in eor_bucket_cleanup() will be run
> multiple times, rather than a lifetime issue.

I read it like this:
The cleanup is only registered on the first creation. The copy never registers. 
But the bucket_destroy calls the pool destroy each time.

> 
> On Thu, May 9, 2019 at 11:28 AM Yann Ylavic  wrote:
>> 
>> No it's not actually, nevermind.
>> 
>> On Thu, May 9, 2019 at 11:24 AM Yann Ylavic  wrote:
>>> 
>>> Hmm, if r->pool gets destroyed by the first eor, the
>>> eor_bucket_cleanup() for the copy should NULLify its b->data at the
>>> same time, so it should be safe no?
>>> 
>>> On Thu, May 9, 2019 at 11:22 AM Plüm, Rüdiger, Vodafone Group
>>>  wrote:
>>>> 
>>>> I think your understanding is correct.
>>>> 
>>>> Regards
>>>> 
>>>> Rüdiger
>>>> 
>>>> 
>>>> C2 General
>>>> 
>>>>> -Ursprüngliche Nachricht-
>>>>> Von: Stefan Eissing 
>>>>> Gesendet: Donnerstag, 9. Mai 2019 11:02
>>>>> An: dev@httpd.apache.org
>>>>> Betreff: eor bucket
>>>>> 
>>>>> Could someone help me to check my understanding of the eor bucket
>>>>> implementation?
>>>>> 
>>>>> If an eor bucket is ever copied, there are 2 buckets with their b->data
>>>>> pointing to the request_rec. Since this is local to the bucket,
>>>>> destroying these 2 will call apr_pool_destroy() twice on the pool.
>>>>> Correct?
>>>>> 
>>>>> -Stefan



Re: eor bucket

2019-05-09 Thread Stefan Eissing



> Am 09.05.2019 um 11:28 schrieb Yann Ylavic :
> 
> No it's not actually, nevermind.

Yeah, I was going back and forth like that myself. Therefore my question. ;)

I am seeing in an uncomitted change a rare occasion where eor_bucket_destroy() 
seems to destroy a pool twice. If that is related to a bucket copy, I have not 
found out yet. But it's one more peculiarity to keep in mind...

Thanks, guys!


> On Thu, May 9, 2019 at 11:24 AM Yann Ylavic  wrote:
>> 
>> Hmm, if r->pool gets destroyed by the first eor, the
>> eor_bucket_cleanup() for the copy should NULLify its b->data at the
>> same time, so it should be safe no?
>> 
>> On Thu, May 9, 2019 at 11:22 AM Plüm, Rüdiger, Vodafone Group
>>  wrote:
>>> 
>>> I think your understanding is correct.
>>> 
>>> Regards
>>> 
>>> Rüdiger
>>> 
>>> 
>>> C2 General
>>> 
>>>> -Ursprüngliche Nachricht-
>>>> Von: Stefan Eissing 
>>>> Gesendet: Donnerstag, 9. Mai 2019 11:02
>>>> An: dev@httpd.apache.org
>>>> Betreff: eor bucket
>>>> 
>>>> Could someone help me to check my understanding of the eor bucket
>>>> implementation?
>>>> 
>>>> If an eor bucket is ever copied, there are 2 buckets with their b->data
>>>> pointing to the request_rec. Since this is local to the bucket,
>>>> destroying these 2 will call apr_pool_destroy() twice on the pool.
>>>> Correct?
>>>> 
>>>> -Stefan



eor bucket

2019-05-09 Thread Stefan Eissing
Could someone help me to check my understanding of the eor bucket 
implementation?

If an eor bucket is ever copied, there are 2 buckets with their b->data 
pointing to the request_rec. Since this is local to the bucket, destroying 
these 2 will call apr_pool_destroy() twice on the pool. Correct?

-Stefan

Re: ApacheCon call for presentations, httpd content

2019-05-08 Thread Stefan Eissing
Sounds excellent. What comes to mind in this regard is
- TLS 1.3 support
- the OCSP stapling situation where we are at the moment not the strongest. 
  We should recommend a persistent cache for that - online docs often mention 
only a memory cache.
  When OCSP responders have outages while we find out cached responses invalid, 
people are out of luck.

- Stefan

> Am 08.05.2019 um 20:17 schrieb Dan Ehrlich :
> 
> I would like to give a presentation on hardening / security if possible. 
> 
> I realize this is broad and a little simple for a conference, but the last 
> extensive Apache Security Book was in 2009. 
> 
> It is in no way ready yet and I am extremely self-conscious, but some 
> possible topics that I have written about here and there and could combine:
> 
> - set many many HTTP security headers (there are 9 you can do in Chrome now)
> - an updated SSLCipherSuite list
> - the importance of using ECDHE keys when possible 
> - how to properly structure your /var/www folder regarding static content, 
> executables, uploads, and downloads. 
> - Using both a reverse proxy firewall along with outbound exfilitration 
> scanning with ModSecurity
> - GeoIP Blocking with the new MaxMind API within Apache2
> - followsymlinks danger and how to remediate 
> - other things 
> - any suggestions ppl have or areas they suggest I research :)
> 
> 
>> On May 8, 2019, at 12:55 PM, jean-frederic clere  wrote:
>> 
>>> On 04/05/2019 11:53, Stefan Eissing wrote:
>>> 
>>>>> Am 02.05.2019 um 16:39 schrieb Daniel Ruggeri :
>>>>> 
>>>>> Personally, I'd like to see a presentation on using mod_md, and perhaps
>>>>> something on the benefits of, and use of, http2 in httpd?
>>> 
>>> If anyone wants to present about that and has questions, I'm happy to help.
>>> 
>>> -Stefan
>>> 
>> 
>> What about HTTP/3 there is https://github.com/ngtcp2/nghttp3, do you
>> plan to work on it?
>> 
>> I have a mod_proxy for tomcat, http/2 or 3 for tomcat, I can do a
>> mod_md/ let's encrypt one for httpd (someone else will do the tomcat one)
>> 
>> -- 
>> Cheers
>> 
>> Jean-Frederic



Re: ApacheCon call for presentations, httpd content

2019-05-08 Thread Stefan Eissing
Hi Jean-Frederic,

no plans for H3, need to grow more arms and another head for that. But who 
knows?

Great that you plan to present mod_md. I am starting to make a version 2 for 
that with ACMEv2 support and an alternate OCSP stapling implementation. Maybe 
that is something to mention as well.

Cheers, Stefan

> Am 08.05.2019 um 19:55 schrieb jean-frederic clere :
> 
> On 04/05/2019 11:53, Stefan Eissing wrote:
>> 
>>> Am 02.05.2019 um 16:39 schrieb Daniel Ruggeri :
>>> 
>>>> Personally, I'd like to see a presentation on using mod_md, and perhaps
>>>> something on the benefits of, and use of, http2 in httpd?
>> 
>> If anyone wants to present about that and has questions, I'm happy to help.
>> 
>> -Stefan
>> 
> 
> What about HTTP/3 there is https://github.com/ngtcp2/nghttp3, do you
> plan to work on it?
> 
> I have a mod_proxy for tomcat, http/2 or 3 for tomcat, I can do a
> mod_md/ let's encrypt one for httpd (someone else will do the tomcat one)
> 
> -- 
> Cheers
> 
> Jean-Frederic



mod_md version 2

2019-05-06 Thread Stefan Eissing
Heya,

the beautiful people at MOSS, Mozilla's Open Source Support, decided to give me 
a grant for Let's Encrypt and Stapling improvements in Apache! Big thanks!

I described what I plan to do here: 
https://github.com/icing/mod_md/wiki/V2Design

There are also github issues for collecting feedback and I pointed people to 
the Apache users mailing list as well.

Besides the support for ACMEv2, which is in-scope of the module, I plan to add 
a new OCSP stapling implementation in the module as well. That may lead to some 
head scratching here and I want to explain my reasoning and, ideally, get 
feedback from you.

- Any new OCSP stapling implementation must, in the 2.4.x release line, live 
along side the existing one. We will, with out backward compatibility 
requirements, never be able to switch that one off in 2.4.x.

- We need a new implementation in the near future. OCSP responder downtimes 
become more threatening for the web, just because everyone goes https: if you 
look at browser statistics.

- The infrastructure is there in mod_md. A curl http client, proxy 
configuration and the dependency on mod_watchdog already exist. Linkage to 
openssl (or its cousins) is there as well.

- OCSP answers will be persisted in mod_md's file system store and shared 
between child processes that way.

- Of course, the plan is to have it inactive by default, as shipped by us. 
Admins need to turn it on. If that can be done per
  vhost, MDomain or globally remains to be discussed.

Cheers,

Stefan

Re: ApacheCon call for presentations, httpd content

2019-05-04 Thread Stefan Eissing


> Am 02.05.2019 um 16:39 schrieb Daniel Ruggeri :
> 
>> Personally, I'd like to see a presentation on using mod_md, and perhaps
>> something on the benefits of, and use of, http2 in httpd?

If anyone wants to present about that and has questions, I'm happy to help.

-Stefan


Re: keep-alive and vary in 304 responses

2019-04-10 Thread Stefan Eissing



> Am 10.04.2019 um 16:34 schrieb Julian Reschke :
> 
> On 10.04.2019 16:10, Stefan Eissing wrote:
>> 
>> 
>>> Am 10.04.2019 um 15:57 schrieb Julian Reschke :
>>> 
>>> On 10.04.2019 14:53, Plüm, Rüdiger, Vodafone Group wrote:
>>>> ...
>>>> Not sure about this. I guess with TE each hop could be different in what 
>>>> it accepts
>>>> and generates. This is different from CE. As far as I understand the 
>>>> accept-encoding header
>>>> is only for CE not for TE.
>>>> ...
>>> 
>>> Right (that would be "TE").
>>> 
>>> That aside, maybe it's time to remind that HTTP/2 doesn't have transfer
>>> codings?
>> 
>> You mean, it does not have "chunked".
> 
> Not my reading...:
> 
> "The only exception to this is the TE header field, which MAY be present
> in an HTTP/2 request; when it is, it MUST NOT contain any value other
> than "trailers"."
> 
> <https://greenbytes.de/tech/webdav/rfc7540.html#rfc.section.8.1.2.2.p.2>

Yeah, your reading is correct. There seems to be no wriggle room about that one.

This discourages efforts to extend transfer-encodings for HTTP/1.1. Using 
HTTP/2 to the client in a reverse proxy is quite common. Those proxies could 
pass transfer-encoded, gzipped resources only by uncompressing them. Bleh!

I think I will make an attempt at improving mod_deflate with a second filter 
and see how complicated that gets. Alternate ideas always welcome. But 
content-encoding it seems to be.

-Stefan

PS. The "keep-alive" warning indicator at redbot.org for testing an Apache 
server is gone. Mark agreed to remove the "Connection: keep-alive" request 
header, so Apache will not generate one in response. All's well.



Re: keep-alive and vary in 304 responses

2019-04-10 Thread Stefan Eissing



> Am 10.04.2019 um 15:57 schrieb Julian Reschke :
> 
> On 10.04.2019 14:53, Plüm, Rüdiger, Vodafone Group wrote:
>> ...
>> Not sure about this. I guess with TE each hop could be different in what it 
>> accepts
>> and generates. This is different from CE. As far as I understand the 
>> accept-encoding header
>> is only for CE not for TE.
>> ...
> 
> Right (that would be "TE").
> 
> That aside, maybe it's time to remind that HTTP/2 doesn't have transfer
> codings?

You mean, it does not have "chunked".

> Best regards, Julian



Re: keep-alive and vary in 304 responses

2019-04-10 Thread Stefan Eissing



> Am 10.04.2019 um 14:07 schrieb Plüm, Rüdiger, Vodafone Group 
> :
> 
> 
> 
> 
> C2 General
> 
>> -Ursprüngliche Nachricht-
>> Von: Yann Ylavic 
>> Gesendet: Mittwoch, 10. April 2019 14:04
>> An: httpd-dev 
>> Betreff: Re: keep-alive and vary in 304 responses
>> 
>> On Wed, Apr 10, 2019 at 1:03 PM Plüm, Rüdiger, Vodafone Group
>>  wrote:
>>> 
 -Ursprüngliche Nachricht-
 Von: Yann Ylavic 
 Gesendet: Mittwoch, 10. April 2019 12:49
 
 Do user-agents support "Transfer-Encoding: gzip, chunked" currently?
 That'd be the best/easier solution I think.
>>> 
>>> But TE is only hop-by-hop isn't it?. This would mean no e2e
>> compression?
>> 
>> A proxy can still forward hop-by-hop things, e.g. mod_proxy forwards
>> "chunked" encoding if no previous handler/input-filter reads the body.
> 
> My understanding was that mod_proxy "dechunks" the body and "chunks" it again.
> It does not pass it transparently.
> This would be a resource intensive job with compression.

I believe that a proxy could de-chunk without removing the gzip. 
But how would our output filter infrastructure treat that? mod_deflate
stays inactive when faced with an existing Content-Encoding, but does not
watch out for Transfer-Encodings already applied. No resource type output
filter does, right?

Interesting topic.

- Stefan










Re: keep-alive and vary in 304 responses

2019-04-10 Thread Stefan Eissing



> Am 10.04.2019 um 12:53 schrieb Plüm, Rüdiger, Vodafone Group 
> :
> 
> 
> 
> 
> C2 General
> 
>> -Ursprüngliche Nachricht-
>> Von: Yann Ylavic 
>> Gesendet: Mittwoch, 10. April 2019 12:49
>> An: httpd-dev 
>> Betreff: Re: keep-alive and vary in 304 responses
>> 
>> On Wed, Apr 10, 2019 at 12:10 PM Stefan Eissing
>>  wrote:
>>> 
>>>> Am 09.04.2019 um 18:48 schrieb Roy T. Fielding :
>>>>> 
>>>>> 2. Validation responses lose the "Vary" header from the
>> unconditional response. This happens on resources where mod_deflate is
>> active.
>>>>> The 200 response without any "if-modified-since" etc. carries
>> "Vary: Accept-Encoding" as it should.
>>>>> The 304 response with "if-modified-since" has no "Vary" header.
>>>>> The code in mod_deflate looks as if it tries to add it in any
>> case, where is it lost?
>>>> 
>>>> That's one of many bugs related to design problems with mod_deflate.
>> Basically,
>>>> it chooses to insert itself into the response handling after the
>> other checks are done,
>>>> which means a 304 doesn't even know yet whether the response would
>> use gzip.
>>>> The solution (which I never found time to implement) is to replace
>> dynamic gzip with
>>>> a transfer encoding generator,
>> 
>> Do user-agents support "Transfer-Encoding: gzip, chunked" currently?
>> That'd be the best/easier solution I think.
> 
> But TE is only hop-by-hop isn't it?. This would mean no e2e compression?
> Also we need to keep in mind similar issues for similar filters e.g. 
> mod_brotli

+1

> Regards
> 
> Rüdiger



Re: keep-alive and vary in 304 responses

2019-04-10 Thread Stefan Eissing



> Am 10.04.2019 um 12:48 schrieb Plüm, Rüdiger, Vodafone Group 
> :
> 
> 
> 
> 
> C2 General
> 
>> -Ursprüngliche Nachricht-
>> Von: Stefan Eissing 
>> Gesendet: Mittwoch, 10. April 2019 12:10
>> An: dev@httpd.apache.org
>> Betreff: Re: keep-alive and vary in 304 responses
>> 
>> 
>>> Am 09.04.2019 um 18:48 schrieb Roy T. Fielding :
>>> 
>>>> 
>>>> 2. Validation responses lose the "Vary" header from the unconditional
>> response. This happens on resources where mod_deflate is active.
>>>> The 200 response without any "if-modified-since" etc. carries "Vary:
>> Accept-Encoding" as it should.
>>>> The 304 response with "if-modified-since" has no "Vary" header.
>>>> The code in mod_deflate looks as if it tries to add it in any case,
>> where is it lost?
>>> 
>>> That's one of many bugs related to design problems with mod_deflate.
>> Basically,
>>> it chooses to insert itself into the response handling after the other
>> checks are done,
>>> which means a 304 doesn't even know yet whether the response would use
>> gzip.
>>> The solution (which I never found time to implement) is to replace
>> dynamic gzip with
>>> a transfer encoding generator, and/or to rewrite the mod_deflate CE
>> encoder to act more
>>> like mod_cache.
>> 
>> Thinking out loud here:
>> 
>> Acting more like mod_cache would then split the deflate output filter
>> into a resource and a protocol type one, I assume?
> 
> IMHO the "secret" of mod_cache is that its filter is also inserted via the
> insert_error_filter hook and not just via the insert_filter hook (like 
> mod_expires and mod_header do).

Thanks for the pointer. The first lines of this actually show:

/* ignore everything except for 5xx errors */
if (r->status < HTTP_INTERNAL_SERVER_ERROR) {
return;
}

So, this is not applied on 304s. And 304 is the only non-200 status that
content-encoding filters are really interested in, I think.

> With mod_deflate this will be more complex as mod_deflate does not add itself.
> It is either added by the core via SetOuputFilter or via mod_filter and they
> all only consider insert_filter and not insert_error_filter. We probably need
> a possibility to tell filters to be added in insert_error_filter.

Could not a deflate *protocol* filter always insert itself and check if
its resource counterpart is present in r->output_filters? This would leave
the application of resource filters unchanged.

Only in error responses, e.g. 304, would the protocol deflate filter not
find its buddy. If the r->output_filter got preserved at the request, it could.

> 
> Regards
> 
> Rüdiger



Re: keep-alive and vary in 304 responses

2019-04-10 Thread Stefan Eissing


> Am 09.04.2019 um 18:48 schrieb Roy T. Fielding :
> 
>> 
>> 2. Validation responses lose the "Vary" header from the unconditional 
>> response. This happens on resources where mod_deflate is active.
>>  The 200 response without any "if-modified-since" etc. carries "Vary: 
>> Accept-Encoding" as it should.
>>  The 304 response with "if-modified-since" has no "Vary" header.
>>  The code in mod_deflate looks as if it tries to add it in any case, where 
>> is it lost?
> 
> That's one of many bugs related to design problems with mod_deflate.  
> Basically,
> it chooses to insert itself into the response handling after the other checks 
> are done,
> which means a 304 doesn't even know yet whether the response would use gzip.
> The solution (which I never found time to implement) is to replace dynamic 
> gzip with
> a transfer encoding generator, and/or to rewrite the mod_deflate CE encoder 
> to act more
> like mod_cache.

Thinking out loud here:

Acting more like mod_cache would then split the deflate output filter into a 
resource and a protocol type one, I assume?

The protocol one would
a. check if its resource counterpart was active
b. or, if the requests status is 304, if it would have been active
add "Vary: Accept-Encoding" if one of those is true

For b. a bit of heuristic seems necessary:
- detect that the resource filter has been added before *)
- assume it applies when content-length is not set
- check content-length otherwise
- perform the other (r->subprocess_env etc.) checks as in resource filter


*) It could detect the resource filter in r->output_filters, however 
ap_send_error_respons() overwrites this with r->proto_output_filters. Seems 
like a good idea to save them for later inspection?


-Stefan

Re: keep-alive and vary in 304 responses

2019-04-10 Thread Stefan Eissing



> Am 10.04.2019 um 09:24 schrieb Mario Brandt :
> 
> On Tue, 9 Apr 2019 at 12:31, Stefan Eissing
>  wrote:
>> 
>> I just did some tests with https://redbot.org/ (the site tester by Mark 
>> Nottingham) against our server and it notifies of 2 things:
>> 
>> 1. The "Keep-Alive" header is deprecated. I tried to "Header unset 
>> Keep-Alive" but that has no effect. Seems to be added very late.
>>   Do we have a way to suppress it?
> 
> That is true for HTTP/2. What about the clients that don't support
> HTTP/2 yet? I wonder if it is possible to suppress that header only if
> HTTP2 or better is used.

Our HTTP/2 implementation does indeed suppress it as it does not honor the 
*KeepAlive* settings that apply to HTTP/1.1

As Apache will only generate "Keep-Alive" response headers if the client 
included a "Connection: keep-alive" in its request, I now regard this as a 
non-issue.

Cheers, Stefan

Re: keep-alive and vary in 304 responses

2019-04-09 Thread Stefan Eissing



> Am 09.04.2019 um 16:03 schrieb Stefan Eissing :
> 
> 
> 
>> Am 09.04.2019 um 15:41 schrieb Stefan Eissing :
>> 
>>> 
>>> Am 09.04.2019 um 13:36 schrieb Stefan Eissing 
>>> :
>>> 
>>> 
>>> 
>>>> Am 09.04.2019 um 13:27 schrieb Eric Covener :
>>>> 
>>>> On Tue, Apr 9, 2019 at 6:31 AM Stefan Eissing
>>>>  wrote:
>>>>> 
>>>>> I just did some tests with https://redbot.org/ (the site tester by Mark 
>>>>> Nottingham) against our server and it notifies of 2 things:
>>>>> 
>>>>> 1. The "Keep-Alive" header is deprecated. I tried to "Header unset 
>>>>> Keep-Alive" but that has no effect. Seems to be added very late.
>>>>> Do we have a way to suppress it?
>>>> 
>>>> Doesn't look like it, maybe a good "help wanted" kind of task if
>>>> anyone is lurking.
>>> 
>>> We rarely ever eat volunteers!
>>> 
>>>>> 2. Validation responses lose the "Vary" header from the unconditional 
>>>>> response. This happens on resources where mod_deflate is active.
>>>>> The 200 response without any "if-modified-since" etc. carries "Vary: 
>>>>> Accept-Encoding" as it should.
>>>>> The 304 response with "if-modified-since" has no "Vary" header.
>>>>> The code in mod_deflate looks as if it tries to add it in any case, where 
>>>>> is it lost?
>>>> 
>>>> bailing here, seems harmless if we add Vary to a 304 even when we
>>>> don't know the original was long enough?
>>>> 
>>>> [Tue Apr 09 07:26:12.183430 2019] [deflate:trace1] [pid 37010:tid
>>>> 123145306648576] mod_deflate.c(590): [client 127.0.0.1:64941] Not
>>>> compressing very small response of 0 bytes
>>> 
>>> Ah, nice! 
>>> 
>>> Maybe if AP_STATUS_IS_HEADER_ONLY(r), the length check should not run?
>> 
>> Meh!
>> 
>>> curl -D xxx -H 'Accept-Encoding: gzip' https://eissing.org
>> HTTP/2 200
>> date: Tue, 09 Apr 2019 13:36:10 GMT
>> server: Apache/2.4.39 (Ubuntu)
>> strict-transport-security: max-age=15768000
>> last-modified: Thu, 10 Aug 2017 11:16:47 GMT
>> etag: "9a6-55664550a25c0-gzip"
>> accept-ranges: bytes
>> cache-control: max-age=3600
>> vary: Accept-Encoding
>> content-encoding: gzip
>> content-length: 1015
>> content-type: text/html
>> 
>>> curl -D xxx -H 'Accept-Encoding: gzip' -H 'If-Modified-Since: Thu, 10 Aug 
>>> 2017 11:16:47 GMT' https://eissing.org
>> HTTP/2 304
>> date: Tue, 09 Apr 2019 13:36:35 GMT
>> server: Apache/2.4.39 (Ubuntu)
>> etag: "9a6-55664550a25c0"
>> cache-control: max-age=3600
>> 
>> Same for HTTP/1.1 requests. And same for https://httpd.apache.org
>> 
>> The output filters are not in play at all. Was it always like this?
> 
> Update: its the short-hand header list that gets written in case of 304 in 
> http_filters.c:1429 pp
> 
> Continuing to investigate what is happening here...

Finally, not being the sharpest today, there is - of course - 
http_protocol.c:1248 which removes all resource filters for "error" responses.

Hmmpf. Makes sense as it keeps complexity from filters. The price being that we 
allow only a very limited set as response headers. Because all else could be 
altered by the filters we did not run. Which loses the "Vary" set by resource 
filters.

Hmm, something to chew on...

-Stefan





Re: keep-alive and vary in 304 responses

2019-04-09 Thread Stefan Eissing



> Am 09.04.2019 um 15:41 schrieb Stefan Eissing :
> 
>> 
>> Am 09.04.2019 um 13:36 schrieb Stefan Eissing :
>> 
>> 
>> 
>>> Am 09.04.2019 um 13:27 schrieb Eric Covener :
>>> 
>>> On Tue, Apr 9, 2019 at 6:31 AM Stefan Eissing
>>>  wrote:
>>>> 
>>>> I just did some tests with https://redbot.org/ (the site tester by Mark 
>>>> Nottingham) against our server and it notifies of 2 things:
>>>> 
>>>> 1. The "Keep-Alive" header is deprecated. I tried to "Header unset 
>>>> Keep-Alive" but that has no effect. Seems to be added very late.
>>>> Do we have a way to suppress it?
>>> 
>>> Doesn't look like it, maybe a good "help wanted" kind of task if
>>> anyone is lurking.
>> 
>> We rarely ever eat volunteers!
>> 
>>>> 2. Validation responses lose the "Vary" header from the unconditional 
>>>> response. This happens on resources where mod_deflate is active.
>>>> The 200 response without any "if-modified-since" etc. carries "Vary: 
>>>> Accept-Encoding" as it should.
>>>> The 304 response with "if-modified-since" has no "Vary" header.
>>>> The code in mod_deflate looks as if it tries to add it in any case, where 
>>>> is it lost?
>>> 
>>> bailing here, seems harmless if we add Vary to a 304 even when we
>>> don't know the original was long enough?
>>> 
>>> [Tue Apr 09 07:26:12.183430 2019] [deflate:trace1] [pid 37010:tid
>>> 123145306648576] mod_deflate.c(590): [client 127.0.0.1:64941] Not
>>> compressing very small response of 0 bytes
>> 
>> Ah, nice! 
>> 
>> Maybe if AP_STATUS_IS_HEADER_ONLY(r), the length check should not run?
> 
> Meh!
> 
>> curl -D xxx -H 'Accept-Encoding: gzip' https://eissing.org
> HTTP/2 200
> date: Tue, 09 Apr 2019 13:36:10 GMT
> server: Apache/2.4.39 (Ubuntu)
> strict-transport-security: max-age=15768000
> last-modified: Thu, 10 Aug 2017 11:16:47 GMT
> etag: "9a6-55664550a25c0-gzip"
> accept-ranges: bytes
> cache-control: max-age=3600
> vary: Accept-Encoding
> content-encoding: gzip
> content-length: 1015
> content-type: text/html
> 
>> curl -D xxx -H 'Accept-Encoding: gzip' -H 'If-Modified-Since: Thu, 10 Aug 
>> 2017 11:16:47 GMT' https://eissing.org
> HTTP/2 304
> date: Tue, 09 Apr 2019 13:36:35 GMT
> server: Apache/2.4.39 (Ubuntu)
> etag: "9a6-55664550a25c0"
> cache-control: max-age=3600
> 
> Same for HTTP/1.1 requests. And same for https://httpd.apache.org
> 
> The output filters are not in play at all. Was it always like this?

Update: its the short-hand header list that gets written in case of 304 in 
http_filters.c:1429 pp

Continuing to investigate what is happening here...

Re: keep-alive and vary in 304 responses

2019-04-09 Thread Stefan Eissing


> Am 09.04.2019 um 13:36 schrieb Stefan Eissing :
> 
> 
> 
>> Am 09.04.2019 um 13:27 schrieb Eric Covener :
>> 
>> On Tue, Apr 9, 2019 at 6:31 AM Stefan Eissing
>>  wrote:
>>> 
>>> I just did some tests with https://redbot.org/ (the site tester by Mark 
>>> Nottingham) against our server and it notifies of 2 things:
>>> 
>>> 1. The "Keep-Alive" header is deprecated. I tried to "Header unset 
>>> Keep-Alive" but that has no effect. Seems to be added very late.
>>>  Do we have a way to suppress it?
>> 
>> Doesn't look like it, maybe a good "help wanted" kind of task if
>> anyone is lurking.
> 
> We rarely ever eat volunteers!
> 
>>> 2. Validation responses lose the "Vary" header from the unconditional 
>>> response. This happens on resources where mod_deflate is active.
>>>  The 200 response without any "if-modified-since" etc. carries "Vary: 
>>> Accept-Encoding" as it should.
>>>  The 304 response with "if-modified-since" has no "Vary" header.
>>>  The code in mod_deflate looks as if it tries to add it in any case, where 
>>> is it lost?
>> 
>> bailing here, seems harmless if we add Vary to a 304 even when we
>> don't know the original was long enough?
>> 
>> [Tue Apr 09 07:26:12.183430 2019] [deflate:trace1] [pid 37010:tid
>> 123145306648576] mod_deflate.c(590): [client 127.0.0.1:64941] Not
>> compressing very small response of 0 bytes
> 
> Ah, nice! 
> 
> Maybe if AP_STATUS_IS_HEADER_ONLY(r), the length check should not run?

Meh!

> curl -D xxx -H 'Accept-Encoding: gzip' https://eissing.org
HTTP/2 200
date: Tue, 09 Apr 2019 13:36:10 GMT
server: Apache/2.4.39 (Ubuntu)
strict-transport-security: max-age=15768000
last-modified: Thu, 10 Aug 2017 11:16:47 GMT
etag: "9a6-55664550a25c0-gzip"
accept-ranges: bytes
cache-control: max-age=3600
vary: Accept-Encoding
content-encoding: gzip
content-length: 1015
content-type: text/html

> curl -D xxx -H 'Accept-Encoding: gzip' -H 'If-Modified-Since: Thu, 10 Aug 
> 2017 11:16:47 GMT' https://eissing.org
HTTP/2 304
date: Tue, 09 Apr 2019 13:36:35 GMT
server: Apache/2.4.39 (Ubuntu)
etag: "9a6-55664550a25c0"
cache-control: max-age=3600

Same for HTTP/1.1 requests. And same for https://httpd.apache.org

The output filters are not in play at all. Was it always like this?

-Stefan *scratches his head*



Re: keep-alive and vary in 304 responses

2019-04-09 Thread Stefan Eissing



> Am 09.04.2019 um 13:27 schrieb Eric Covener :
> 
> On Tue, Apr 9, 2019 at 6:31 AM Stefan Eissing
>  wrote:
>> 
>> I just did some tests with https://redbot.org/ (the site tester by Mark 
>> Nottingham) against our server and it notifies of 2 things:
>> 
>> 1. The "Keep-Alive" header is deprecated. I tried to "Header unset 
>> Keep-Alive" but that has no effect. Seems to be added very late.
>>   Do we have a way to suppress it?
> 
> Doesn't look like it, maybe a good "help wanted" kind of task if
> anyone is lurking.

We rarely ever eat volunteers!

>> 2. Validation responses lose the "Vary" header from the unconditional 
>> response. This happens on resources where mod_deflate is active.
>>   The 200 response without any "if-modified-since" etc. carries "Vary: 
>> Accept-Encoding" as it should.
>>   The 304 response with "if-modified-since" has no "Vary" header.
>>   The code in mod_deflate looks as if it tries to add it in any case, where 
>> is it lost?
> 
> bailing here, seems harmless if we add Vary to a 304 even when we
> don't know the original was long enough?
> 
> [Tue Apr 09 07:26:12.183430 2019] [deflate:trace1] [pid 37010:tid
> 123145306648576] mod_deflate.c(590): [client 127.0.0.1:64941] Not
> compressing very small response of 0 bytes

Ah, nice! 

Maybe if AP_STATUS_IS_HEADER_ONLY(r), the length check should not run?

> --
> Eric Covener
> cove...@gmail.com



keep-alive and vary in 304 responses

2019-04-09 Thread Stefan Eissing
I just did some tests with https://redbot.org/ (the site tester by Mark 
Nottingham) against our server and it notifies of 2 things:

1. The "Keep-Alive" header is deprecated. I tried to "Header unset Keep-Alive" 
but that has no effect. Seems to be added very late. 
   Do we have a way to suppress it?

2. Validation responses lose the "Vary" header from the unconditional response. 
This happens on resources where mod_deflate is active.
   The 200 response without any "if-modified-since" etc. carries "Vary: 
Accept-Encoding" as it should.
   The 304 response with "if-modified-since" has no "Vary" header.
   The code in mod_deflate looks as if it tries to add it in any case, where is 
it lost?

Help in any of the two issues above is appreciated!

-Stefan

Re: [users@httpd] Error while build apache 2.4.39 using CMake on Window machine

2019-04-04 Thread Stefan Eissing
I made the change in trunk with revision r1856910 and will propose it for 
backport to 2.4.x

> Am 04.04.2019 um 11:47 schrieb Rathore, Rajendra :
> 
> Hi Stefan,
> 
> Thanks for the help, Do I need to raise an issue with Apache Http server to 
> update that file or we have any open issue like below.
> 
> Thanks and Regards,
> Rajendra Rathore
> 9922701491
> 
> -Original Message-
> From: Stefan Eissing  
> Sent: 04 April 2019 03:15 PM
> To: us...@httpd.apache.org
> Subject: Re: [users@httpd] Error while build apache 2.4.39 using CMake on 
> Window machine
> 
> External email from: users-return-118594-rarathore=ptc@httpd.apache.org
> 
> In ./CMakeLists.txt, line 423
> 
>  modules/http2/h2_switch.c  modules/http2/h2_ngn_shed.c 
> 
> change to
> 
>  modules/http2/h2_switch.c
> 
> 
> 
> 
>> Am 04.04.2019 um 11:42 schrieb Rathore, Rajendra :
>> 
>> Hi Stefan,
>> 
>> Thanks for the quick response, please let me know what entry should I need 
>> to remove?
>> 
>> Thanks and Regards,
>> Rajendra Rathore
>> 9922701491
>> 
>> -Original Message-
>> From: Stefan Eissing 
>> Sent: 04 April 2019 03:10 PM
>> To: us...@httpd.apache.org
>> Subject: Re: [users@httpd] Error while build apache 2.4.39 using CMake 
>> on Window machine
>> Importance: High
>> 
>> External email from: 
>> users-return-118592-rarathore=ptc@httpd.apache.org
>> 
>> The source file is gone. It needs to be removed from CMakeLists.txt. Sorry 
>> about the confusion.
>> 
>>> Am 04.04.2019 um 11:34 schrieb Rathore, Rajendra :
>>> 
>>> Hi Team,
>>> 
>>> While building apache 2.4.39 using CMake command, I face below issue
>>> 
>>> CMake Error at CMakeLists.txt:761 (ADD_LIBRARY):
>>> Cannot find source file:
>>> 
>>>   modules/http2/h2_ngn_shed.c
>>> 
>>> Tried extensions .c .C .c++ .cc .cpp .cxx .m .M .mm .h .hh .h++ .hm 
>>> .hpp  .hxx .in .txx
>>> 
>>> 
>>> CMake Error: CMake can not determine linker language for target: 
>>> mod_http2 CMake Error: Cannot determine link language for target 
>>> "mod_http2".
>>> CMake Error:
>>> Error evaluating generator expression:
>>> 
>>>   $
>>> 
>>> TARGET_PDB_FILE is not supported by the target linker.
>>> 
>>> Can someone please help me to fix the issue, since it contains critical 
>>> security fix and I need to incorporate the changes.
>>> 
>>> Thanks and Regards,
>>> Rajendra Rathore
>>> 9922701491
>> 
>> 
>> -
>> To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
>> For additional commands, e-mail: users-h...@httpd.apache.org
>> 
>> 
>> 
>> -
>> To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
>> For additional commands, e-mail: users-h...@httpd.apache.org
>> 
> 
> 
> -
> To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
> For additional commands, e-mail: users-h...@httpd.apache.org
> 
> 
> 
> -
> To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
> For additional commands, e-mail: users-h...@httpd.apache.org
> 



Re: [VOTE] Release httpd-2.4.39

2019-03-29 Thread Stefan Eissing



> Am 29.03.2019 um 13:55 schrieb Eric Covener :
> 
> On Wed, Mar 27, 2019 at 11:10 AM Daniel Ruggeri  wrote:
>> 
>> Hi, all;
>>Please find below the proposed release tarball and signatures:
>> https://dist.apache.org/repos/dist/dev/httpd/
>> 
>> I would like to call a VOTE over the next few days to release this
>> candidate tarball as 2.4.39:
>> [ x] +1: It's not just good, it's good enough!
>> [ ] +0: Let's have a talk.
>> [ ] -1: There's trouble in paradise. Here's what's wrong.
> 
> +1 on AIX/PPC64/xlc w/ openssl1.1.1b
> 
> Caveats:
> - mod_http2 needed a trivial build change that I submitted on GH.

The use of strcmp() has been there since the beginning of time. I am
not sure what changed in your setup that messed up your linking.

> - My perl on AIX is aging poorly (perl linked w/ old openssl) and has
> a number of failures but none that are too alarming
> 
> 
> t/ab/base.t   (Wstat: 0 Tests: 5 Failed: 4)
>  Failed tests:  1-4
> (Not a regression in .39 -- --enable-static-support is not working well on 
> AIX)
> 
> perl ssl problems I am used to seeing:
> 
> t/security/CVE-2009-3555.t(Wstat: 65280 Tests: 0 Failed: 0)
>  Non-zero exit status: 255
>  Parse errors: Bad plan.  You planned 4 tests but ran 0.
> t/ssl/ocsp.t  (Wstat: 0 Tests: 3 Failed: 2)
>  Failed tests:  2-3
> t/ssl/proxy.t (Wstat: 0 Tests: 172 Failed: 64)
>  Failed tests:  3-7, 114-172



Re: [VOTE] Release httpd-2.4.39

2019-03-28 Thread Stefan Eissing


> Am 27.03.2019 um 16:09 schrieb Daniel Ruggeri :
> 
> Hi, all;
>   Please find below the proposed release tarball and signatures:
> https://dist.apache.org/repos/dist/dev/httpd/
> 
> I would like to call a VOTE over the next few days to release this candidate 
> tarball as 2.4.39:
> [ ] +1: It's not just good, it's good enough!
> [ ] +0: Let's have a talk.
> [ ] -1: There's trouble in paradise. Here's what's wrong.

+1: all running fine
- h2+md tests on MacOS 10.14.4
- h2 tests on ubuntu 16.04 LTS
- h2fuzzing on ubuntu 16.04 LTS

Thanks for RMing!

Re: [NOTICE] Intent to T 2.4.39

2019-03-26 Thread Stefan Eissing
Nothing like a members meeting to sneak in some test changes!

> Am 26.03.2019 um 18:55 schrieb Joe Orton :
> 
> On Tue, Mar 26, 2019 at 02:39:02PM -, Daniel Ruggeri wrote:
>> I'll assume the absence of shouts to be a good sign :-)
>> 
>> I'll T in about 24 hours from now.
> 
> Smoke tests of 2.4.x are looking good here, goferit!
> 
> Regards, Joe



Re: svn commit: r1855743 - /httpd/httpd/trunk/server/util.c

2019-03-18 Thread Stefan Eissing
The question is, should our buildbot detect those things?

> Am 18.03.2019 um 13:46 schrieb Ruediger Pluem :
> 
> 
> 
> On 03/18/2019 01:42 PM, Ruediger Pluem wrote:
>> 
>> 
>> On 03/18/2019 01:22 PM, Eric Covener wrote:
>>> I just found that this is a compile error on my mac w/ maintainer mode
>>> due to __attribute__((nonnull))
>>> util.c:576:10: error: nonnull parameter 'name' will evaluate to 'true'
>>> on first encounter [-Werror,-Wpointer-bool-conversion]
>> 
>> Hm. I guess my RedHat 6 compiler is too old and no maintainer mode used. 
>> Hence I did not catch that.
>> So should we just revert r1855743 (unfortunately in 2.4.x then as well :-( ) 
>> and put the NULL checking
>> burden on the caller? I guess in the specific case here where it caused the 
>> segfault we are safe
>> as r1855744 catches that.
> 
> Ok, r1855755 is even better :-)
> Voted on it. Hopefully everything is fine now.
> 
> Regards
> 
> Rüdiger
> 
> 



Re: svn commit: r1855737 - in /httpd/httpd/branches/2.4.x: CHANGES docs/manual/mod/core.xml include/ap_mmn.h include/http_core.h include/httpd.h server/core.c server/request.c server/util.c

2019-03-18 Thread Stefan Eissing



> Am 18.03.2019 um 10:32 schrieb Ruediger Pluem :
> 
> 
> 
> On 03/18/2019 09:49 AM, ic...@apache.org wrote:
>> Author: icing
>> Date: Mon Mar 18 08:49:59 2019
>> New Revision: 1855737
>> 
>> URL: http://svn.apache.org/viewvc?rev=1855737=rev
>> Log:
>> Merge of r1855705 from trunk:
>> 
>> core: merge consecutive slashes in the path
>> 
>> 
>> Modified:
>>httpd/httpd/branches/2.4.x/CHANGES
>>httpd/httpd/branches/2.4.x/docs/manual/mod/core.xml
>>httpd/httpd/branches/2.4.x/include/ap_mmn.h
>>httpd/httpd/branches/2.4.x/include/http_core.h
>>httpd/httpd/branches/2.4.x/include/httpd.h
>>httpd/httpd/branches/2.4.x/server/core.c
>>httpd/httpd/branches/2.4.x/server/request.c
>>httpd/httpd/branches/2.4.x/server/util.c
> 
> Unfortunately I just detected that this will always SEGFAULT with CONNECT 
> methods without
> r1855743, r1855744 :-(
> 
> Can we have two people review r1855743, r1855744 quickly to get this fixed?

Just added my vote for it to STATUS.

> Regards
> 
> Rüdiger
> 



Re: h2 backport proposal

2019-03-13 Thread Stefan Eissing
> Am 13.03.2019 um 15:41 schrieb Yann Ylavic :
> 
> Got Jim's and my votes, could be merged now ;)
> Please note that the CHANGES entries from the patch are slightly mixed
> with unrelated changes, should be cleaned up when backporting.
> Nice changes otherwise, thanks!

Thanks, Yann! Will do!


> On Wed, Mar 13, 2019 at 2:05 PM Stefan Eissing
>  wrote:
>> 
>> Proposal updated in 1855417.
>> 
>>> Am 13.03.2019 um 13:46 schrieb Stefan Eissing 
>>> :
>>> 
>>> 
>>>> Am 13.03.2019 um 13:32 schrieb Yann Ylavic :
>>>> 
>>>> On Wed, Mar 13, 2019 at 12:55 PM Yann Ylavic  wrote:
>>>>> 
>>>>> On Wed, Mar 13, 2019 at 12:49 PM Stefan Eissing
>>>>>  wrote:
>>>>>> 
>>>>>> It would be nice if 2 people could find the time to have a look at it.
>>>>> 
>>>>> I'm currently reviewing it, looks good so far, nice!
>>>> 
>>>> Looks like the proposal reverts r1852989, normal?
>>> 
>>> Nicely caught! I forgot to bring this over to github, so my recent merge 
>>> over did revert it.
>>> 
>>> Re-added to trunk in: r1855411. Will add to backport proposal.
>>> 
>>> Thanks, Yann!
>>> 
>> 



Re: h2 backport proposal

2019-03-13 Thread Stefan Eissing
Proposal updated in 1855417.

> Am 13.03.2019 um 13:46 schrieb Stefan Eissing :
> 
> 
>> Am 13.03.2019 um 13:32 schrieb Yann Ylavic :
>> 
>> On Wed, Mar 13, 2019 at 12:55 PM Yann Ylavic  wrote:
>>> 
>>> On Wed, Mar 13, 2019 at 12:49 PM Stefan Eissing
>>>  wrote:
>>>> 
>>>> It would be nice if 2 people could find the time to have a look at it.
>>> 
>>> I'm currently reviewing it, looks good so far, nice!
>> 
>> Looks like the proposal reverts r1852989, normal?
> 
> Nicely caught! I forgot to bring this over to github, so my recent merge over 
> did revert it.
> 
> Re-added to trunk in: r1855411. Will add to backport proposal.
> 
> Thanks, Yann!
> 



Re: h2 backport proposal

2019-03-13 Thread Stefan Eissing


> Am 13.03.2019 um 13:32 schrieb Yann Ylavic :
> 
> On Wed, Mar 13, 2019 at 12:55 PM Yann Ylavic  wrote:
>> 
>> On Wed, Mar 13, 2019 at 12:49 PM Stefan Eissing
>>  wrote:
>>> 
>>> It would be nice if 2 people could find the time to have a look at it.
>> 
>> I'm currently reviewing it, looks good so far, nice!
> 
> Looks like the proposal reverts r1852989, normal?

Nicely caught! I forgot to bring this over to github, so my recent merge over 
did revert it.

Re-added to trunk in: r1855411. Will add to backport proposal.

Thanks, Yann!



h2 backport proposal

2019-03-13 Thread Stefan Eissing
There is a mod_http2/mod_proxy_http2 backport proposal in 2.4.x/STATUS
that accumulates this years changes from the github version. This
includes several bugfixes, 2 of which from Michael Kaufmann (Thanks!), 
for issues reported:

  *) mod_http2: HEAD requests to some module such as mod_cgid caused the stream 
to
 terminate improperly and cause a HTTP/2 PROTOCOL_ERROR. 
 Fixes <https://github.com/icing/mod_h2/issues/167>. [Michael Kaufmann]
  *) mod_http2: when SSL renegotiation is inhibited and a 403 ErrorDocument is
 in play, the proper HTTP/2 stream reset did not trigger with 
H2_ERR_HTTP_1_1_REQUIRED.
 Fixed. [Michael Kaufmann] 
  *) mod_proxy_http2: changed mod_proxy_http2 implementation and fixed several 
bugs which
 resolve PR63170. The proxy module does now a single h2 request on the 
(reused)
 connection and returns. [Stefan Eissing]
  *) mod_http2: enable re-use of slave connections again. Fixed slave connection
 keepalives counter. [Stefan Eissing]

Two features have also been implemented:

  *) mod_http2: new configuration directive: ```H2Padding numbits``` to control 
 padding of HTTP/2 payload frames. 'numbits' is a number from 0-8,
 controlling the range of padding bytes added to a frame. The actual number
 added is chosen randomly per frame. This applies to HEADERS, DATA and 
PUSH_PROMISE
 frames equally. The default continues to be 0, e.g. no padding. [Stefan 
Eissing] 
  *) mod_http2: Configuration directives H2Push and H2Upgrade can now be 
specified per 
 Location/Directory, e.g. disabling PUSH for a specific set of resources. 
[Stefan Eissing]

For the H2Padding, I also wrote a blog at: 
https://icing.github.io/mod_h2/padding.html

It would be nice if 2 people could find the time to have a look at it. 

Cheers, Stefan

Re: Anyone interested in a freelance opportunity?

2019-02-22 Thread Stefan Eissing


> Am 22.02.2019 um 12:03 schrieb Ruediger Pluem :
> 
> 
> 
> On 02/21/2019 12:46 AM, Daniel Ruggeri wrote:
>> Hi, all;
>> I was approached to see if I would be interested/willing to work on code to 
>> support encrypted client keys for the proxy.
> 
> You mean encrypted private keys for SSL client authentication?
> You might remember that discussion from 2013 then where you took part:
> 
> https://lists.apache.org/thread.html/5d4fbc62cb07a3550af4f516d007973c385389cace202d217f6b74c1@1384351589@%3Cdev.httpd.apache.org%3E

Interesting. Thanks, Rüdiger.

In mod_md, there is no mechanism besides file permissions to protect private 
keys of server certificates. However, new keys, generated  as less-privileged 
user, are stored encrypted. When the server reloads and copies them into a 
"root" form they are converted to unencrypted. The passphrase sits in memory 
during this time, because.

Generic security scenarios where the attacker gets root access to file system / 
memory rapidly become unconstructive, I find. One needs to focus on a more 
specific scenarios and requirements to get anywhere.

-Stefan

Re: Anyone interested in a freelance opportunity?

2019-02-21 Thread Stefan Eissing



> Am 21.02.2019 um 00:46 schrieb Daniel Ruggeri :
> 
> Hi, all;
> I was approached to see if I would be interested/willing to work on code to 
> support encrypted client keys for the proxy. Unfortunately, I had to pass 
> since I just don't have the time, but figured I'd reach out here to see if 
> anyone here has the time/expertise/interest.
> 
> I know it's an odd thing to ask, but thought it's worth bringing up because 
> I'd personally love to see this functionality :-)
> 
> Feel free to reply directly to me if you don't want to share with the list.

It's no secret that I sometimes take money for developing free software. I have 
some openings later this year. If this fits the time frame and going via a 
German company is not weird, feel free to forward my name and email.

Cheers,

Stefan

PS. I do not find it weird at all to ask here. I see really no difference 
between an employee intended to put time into open source vs. a person hired 
for a certain development in an open source project. And I do not feel that we 
are competing here, either. That'd be a different thing indeed.



Re: Regression?

2019-02-19 Thread Stefan Eissing



> Am 18.02.2019 um 23:55 schrieb Gregg Smith :
> 
> When setting a header it used to set the header case-sensitive as configured. 
> Now with 2.4.38 it sets in all lower case. Regression?
> 
> Header always set X-Xss-Protection "1; mode=block"
> Result;
> 2.4.37: X-Xss-Protection: 1; mode=block
> 2.4.38: x-xss-protection: 1; mode=block

You are not using a HTTP/2 client by any chance?

> If I'm reading the RFC correctly, sensitivity doesn't matter when parsing the 
> header but the 2.4 docs show it outputting as configured as 2.4 has been 
> prior to .38.
> 
> Cheers
> 
> G



Re: svn commit: r1853631 - /httpd/httpd/trunk/modules/md/mod_md_config.c

2019-02-18 Thread Stefan Eissing
Ah, ok. I see. Thanks!

> Am 18.02.2019 um 14:24 schrieb Joe Orton :
> 
> On Mon, Feb 18, 2019 at 01:23:53PM +0100, ste...@eissing.org wrote:
>> Would that not effectively relocate the directory on a server upgrade 
>> and cause all existing certificates to no longer be found?
> 
> If the statedir is not the same as the serverroot, then yes, and I 
> wouldn't propose to change the behaviour in 2.4.x for exactly that 
> reason.
> 
> Regards, Joe
> 
> 
>> 
>>> Am 15.02.2019 um 11:09 schrieb jor...@apache.org:
>>> 
>>> Author: jorton
>>> Date: Fri Feb 15 10:09:53 2019
>>> New Revision: 1853631
>>> 
>>> URL: http://svn.apache.org/viewvc?rev=1853631=rev
>>> Log:
>>> * modules/md/mod_md_config.c (md_mod_conf_get): Use state-dir-relative
>>> default base_dir.
>>> 
>>> Modified:
>>>   httpd/httpd/trunk/modules/md/mod_md_config.c
>>> 
>>> Modified: httpd/httpd/trunk/modules/md/mod_md_config.c
>>> URL: 
>>> http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/md/mod_md_config.c?rev=1853631=1853630=1853631=diff
>>> ==
>>> --- httpd/httpd/trunk/modules/md/mod_md_config.c (original)
>>> +++ httpd/httpd/trunk/modules/md/mod_md_config.c Fri Feb 15 10:09:53 2019
>>> @@ -54,10 +54,18 @@
>>> 
>>> #define DEF_VAL (-1)
>>> 
>>> +#ifndef MD_DEFAULT_BASE_DIR
>>> +#define MD_DEFAULT_BASE_DIR "md"
>>> +#endif
>>> +
>>> /* Default settings for the global conf */
>>> static md_mod_conf_t defmc = {
>>>NULL,
>>> -"md",
>>> +#if AP_MODULE_MAGIC_AT_LEAST(20180906, 2)
>>> +NULL, /* apply default state-dir-relative */
>>> +#else
>>> +MD_DEFAULT_BASE_DIR,
>>> +#endif
>>>NULL,
>>>NULL,
>>>80,
>>> @@ -113,6 +121,11 @@ static md_mod_conf_t *md_mod_conf_get(ap
>>>mod_md_config->mds = apr_array_make(pool, 5, sizeof(const md_t *));
>>>mod_md_config->unused_names = apr_array_make(pool, 5, sizeof(const 
>>> md_t *));
>>> 
>>> +#if AP_MODULE_MAGIC_AT_LEAST(20180906, 2)
>>> +mod_md_config->base_dir = ap_state_dir_relative(pool,
>>> +
>>> MD_DEFAULT_BASE_DIR);
>>> +#endif
>>> +
>>>apr_pool_cleanup_register(pool, NULL, cleanup_mod_config, 
>>> apr_pool_cleanup_null);
>>>}
>>> 
>>> 
>>> 
>> 



Re: svn commit: r1853407 - in /httpd/httpd/trunk/modules: http2/mod_proxy_http2.c proxy/mod_proxy_http.c

2019-02-14 Thread Stefan Eissing
Yann,

The new v3 patch runs without any problems in my test setups. Nice work!

As to the errors you see: that seems to be the output parsing for nghttp
in the test classes that needs some more work. On your systems, DATA
packages arrive different than my MacOS had seen so far. That messed
up the response body compare.

Working things out on a ubuntu image now. Will let you know when I
have something for you to verify.

Cheers,
Stefan

> Am 13.02.2019 um 18:55 schrieb Yann Ylavic :
> 
> Thanks Stefan,
> 
> I think the "400" issue is fixed (r1853518, added to backport
> proposal), but two tests keep failing for in the test suite, namely
> test_004_post and test_500_proxy. They fail with or without my changes
> (trunk and 2.4.x), so I don't think it's related (mod_proxy does not
> seem to be reached for failing requests)..
> 
> My system is debian/openssl-1.1.1a, dunno if openssl version is
> related but forcing "SSLProtocol TLSv1.2" didn't help.
> The attached tarball contains an "output.log" (from: make test 2>&1
> |tee output.log), "mod_h2-test.diff" which is the configuration
> changes I ran with (mainly trace6 with dumpio, and a small/unrelated
> ProxyPass "fix" for the trailing slash), and finally the error_log
> (trace6/dumpio).
> 
> HTH...



Re: svn commit: r1853407 - in /httpd/httpd/trunk/modules: http2/mod_proxy_http2.c proxy/mod_proxy_http.c

2019-02-12 Thread Stefan Eissing
Ok, took the opportunity to add that to my pytest suite in mod-h2.

You need a local "pytest", "curl" and "nghttp" executable. If you checkout 
https://github.com/icing/mod_h2 master, run the configure:

> autoreconf -i
> automake
> autoconf
> ./configure --with-apxs=/opt/apache-2.4.x/bin/apxs --enable-werror
> make
> make test

("/opt/apache-2.4.x" is where I install my locally build 2.4.x branch. Adjust 
to your needs.)

You should see, on an unpatched 2.4.x
= 
test session starts platform darwin -- Python 2.7.10, pytest-3.0.7, py-1.4.33, 
pluggy-0.4.0
rootdir: /Users/sei/projects/mod-h2/test/e2e, inifile:
collected 86 items

test_001_httpd_alive.py ..
test_002_curl_basics.py ..
test_003_get.py .
test_004_post.py 
test_100_conn_reuse.py .
test_101_ssl_reneg.py ...
test_102_require.py ..
test_103_upgrade.py ...
test_200_header_invalid.py ...
test_201_header_conditional.py ..
test_202_trailer.py 
test_300_interim.py ...
test_400_push.py ...
test_401_early_hints.py ..
test_500_proxy.py ...

= 86 passed 
in 23.13 seconds 

On a 2.4.x installed with the proposed proxy patches, the test_500_24 fails 
with a 400 response code from the server.
You can just run that test case via

> (cd test/e2e && pytest -k 500_24)

Hope this helps.

-Stefan

> Am 12.02.2019 um 13:59 schrieb Yann Ylavic :
> 
> OK, thanks Stefan, will look at it.
> 
> 
> On Tue, Feb 12, 2019 at 11:12 AM ste...@eissing.org  
> wrote:
>> 
>> Yann,
>> 
>> this works fine in my tests on trunk. However on 2.4.x I get often errors 
>> when uploading data without content-length.
>> 
>> Scenario from the h2 test suite, HTTP/2 on the front, old skool HTTP/1.1 
>> proxy to itself:
>> 
>>> while true; do nghttp -v --expect-continue --data=gen/tmp/updata 
>>> -H'Content-Type: multipart/form-data; boundary=DSAJKcd9876' 
>>> --no-content-length http://test.example.org:12345/proxy/upload.py; done | 
>>> fgrep 400
>> 
>> [  0.009] recv (stream_id=13) :status: 400
>> 400 Bad Request
>> [  0.007] recv (stream_id=13) :status: 400
>> 400 Bad Request
>> ...
>> 
>> 
>> /etc/hosts
>> 
>> 127.0.0.1   test.example.orgtest
>> 
>> httpd config:
>> 
>>ProxyPass "/proxy" "balancer://http-local"
>>ProxyPassReverse "/proxy" "balancer://http-local"
>> 
>>
>>BalancerMember "http://127.0.0.1:12345; hcmethod=GET hcuri=/ ttl=4
>>
>> 
>> 
>>> cat gen/tmp/updata
>> --DSAJKcd9876
>> Content-Disposition: form-data; name="xxx"; filename="x"
>> Content-Type: text/plain
>> 
>> testing mod_h2
>> --DSAJKcd9876
>> Content-Disposition: form-data; name="file"; filename="data-1k"
>> Content-Type: application/octet-stream
>> Content-Transfer-Encoding: binary
>> 
>> 012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678
>> 012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678
>> 012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678
>> 012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678
>> 012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678
>> 012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678
>> 012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678
>> 012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678
>> 012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678
>> 012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678
>> 
>> --DSAJKcd9876--
>> 
>> 
>> 
>>> Am 11.02.2019 um 22:55 schrieb yla...@apache.org:
>>> 
>>> Author: ylavic
>>> Date: Mon Feb 11 21:55:43 2019
>>> New Revision: 1853407
>>> 
>>> URL: http://svn.apache.org/viewvc?rev=1853407=rev
>>> Log:
>>> mod_proxy_http: rework the flushing strategy when forwarding the request 
>>> body.
>>> 
>>> Since the forwarding of 100-continue (end to end) in r1836588, we depended 
>>> on
>>> reading all of the requested HUGE_STRING_LEN bytes to avoid the flushes, but
>>> this is a bit fragile.
>>> 
>>> This commit introduces the new stream_reqbody_read() function which will 
>>> try a
>>> nonblocking read first and, if it fails with EAGAIN, will flush on the 
>>> backend
>>> side before blocking for the next client side read.
>>> 
>>> We can then use it in stream_reqbody_{chunked,cl}() to flush client 
>>> forwarded
>>> data only when necessary. This both allows "optimal" flushing and simplifies
>>> code (note that spool_reqbody_cl() also makes use of the new function but 
>>> not
>>> its nonblocking/flush functionality, thus only for consistency with the two
>>> 

Re: Prefork MPM and mod_watchdog

2019-01-31 Thread Stefan Eissing



> Am 27.01.2019 um 14:40 schrieb Rainer Jung :
> 
> - as soon as I enable mod_watchdog only (but not the above modules that would 
> use it), the hangs start to happen every now and then.

Hmm, that sounds strange. As I understood the code, none of the parent/child 
watchdogs would be active then. So, its child_init should not do anything 
either.

But, if one of the proxy monitors runs against the server itself, could it 
connect against the child process it (the watchdog) is running on?

-Stefan

http workshop

2019-01-28 Thread Stefan Eissing
The HTTP Workshop is returning 2019 on April 2-4 in Amsterdam 
(https://github.com/httpworkshop/workshop2019). While I attended the last three 
shops(?), I think it would be a good opportunity for someone else from the team 
to go there and meet some smart and friendly people from the HTTP world.

The HTTP WS organisers expressed the wish to have someone from "Apache" 
present. Anyone interested? Could also be someone from another HTTP related 
Apache project, of course.

Cheers,

Stefan


Re: Apache 0-day / apache-uaf / use after free bugs

2019-01-22 Thread Stefan Eissing
Thanks for the update, Stefan!

> Am 22.01.2019 um 13:42 schrieb Stefan Sperling :
> 
> On Tue, Jan 22, 2019 at 01:31:43PM +0100, Rainer Jung wrote:
>> Here's the response we have compiled from Daniel, Stefan and others:
>> 
>> https://bz.apache.org/bugzilla/show_bug.cgi?id=63098
> 
> FYI, I have disabled pool debugging in OpenBSD's port of APR.
> We are now using Yann's patch to force the default allocator to
> call free(3) when APR pools are cleared:
> https://marc.info/?l=openbsd-ports-cvs=154815812713288=2
> 
> This change only affects OpenBSD -current.
> I do not plan to backport a patch to the OpenBSD 6.4 release.
> We have had no reports indicating that http2 was crashing on OpenBSD.
> The likely reason is that nobody is actually running such a setup.
> If people intend to run such a setup, they should use -current for now,
> or wait until OpenBSD 6.5 is released.



Re: Apache 0-day / apache-uaf / use after free bugs

2019-01-22 Thread Stefan Eissing
Thanks! I also wrote about the h2 related parts at 
https://icing.github.io/mod_h2/pool-debugging.html

> Am 22.01.2019 um 13:31 schrieb Rainer Jung :
> 
> Am 22.01.2019 um 10:33 schrieb Daniel Gruno:
>> On 1/22/19 8:09 AM, Stefan Priebe - Profihost AG wrote:
>>> Hi,
>>> 
>>> in twitter and other social media channels they're talking about a
>>> current apache 0 day:
>>> https://twitter.com/i/web/status/1087593706444730369
>>> 
>>> which wasn't handled / isn't currently fixed.
>>> 
>>> Some details are here:
>>> https://github.com/hannob/apache-uaf
>>> 
>>> If this is true there will be exploits soon. Is there anything planned?
>>> Does 2.4.38 fix those issues?
>>> 
>>> Greets,
>>> Stefan
>>> 
>> Hi Stefan, and good morning.
>> I figured I should write something to calm people that might be concerned.
>> I will reply in length in a while (coffee is needed first), it takes time to 
>> write a proper response that explains our processes and considerations with 
>> issues like this, especially when people start hyping the matter. Such is 
>> social media, I guess.
>> Until then, I will say quickly that we do not at present consider this 
>> something you should be alarmed about. Boring elaboration to follow in a 
>> while when I have compiled it :)
>> With regards,
>> Daniel, speaking as just a normal committer.
> 
> Here's the response we have compiled from Daniel, Stefan and others:
> 
> https://bz.apache.org/bugzilla/show_bug.cgi?id=63098
> 
> Regards,
> 
> Rainer



Re: svn commit: r1850745 - /httpd/httpd/trunk/modules/filters/config.m4

2019-01-16 Thread Stefan Eissing



> Am 16.01.2019 um 03:33 schrieb William A Rowe Jr :
> 
> On Tue, Jan 15, 2019 at 8:37 AM Jim Jagielski  wrote:
> 
> > On Jan 15, 2019, at 9:21 AM, Eric Covener  wrote:
> > 
> > On Tue, Jan 15, 2019 at 9:14 AM Jim Jagielski  wrote:
> >> 
> >> On Jan 9, 2019, at 7:41 PM, William A Rowe Jr  wrote:
> >> 
> >> Hi Jim,
> >> 
> >> Does CFLAGS -std=c99 solve your issue? It seems to work here. I'm building 
> >> on the Fedora 29, largely frozen end-of-july. Reverting the patch below 
> >> and toggling -std=c89 to -std=c99 in configure.in building all but two 
> >> modules from trunk is building clean, and results in this command for 
> >> error checking;
> >> /usr/lib64/apr-1/build/libtool --silent --mode=compile gcc  -pthread 
> >> -std=c99 -Werror -Wall -Wstrict-prototypes -Wmissing-prototypes 
> >> -Wmissing-declarations -Wdeclaration-after-statement -Wpointer-arith 
> >> -Wformat -Wformat-security -Wunused -DLINUX -D_REENTRANT -D_GNU_SOURCE 
> >> -DAP_DEBUG
> >> 
> >> Is it reasonable to enforce c99 limitations at this late date? I'm not 
> >> suggesting we change the general builds from c89 in the 2.4 branch, but 
> >> that is something we might want to consider for trunk, 20 years out.
> >> 
> >> 
> >> Personally, I'd be fine allowing c99 in both 2.4 and trunk, considering 
> >> that we are in 2019 already :)
> >> 
> >> Any platform that lacks a c99 compatible CC likely doesn't build anyway.
> > 
> > As a binary distributor, even though a C99 compiler may be available
> > on platform X, it might not be in use.  Wouldn't love seeing it in
> > 2.4.
> 
> I'm not proposing a change for 2.4... but I wouldn't oppose it either :)
> 
> Allowing c99 for trunk would make backporting to 2.4 (which would stay c89) 
> possibly more difficult. This is either a good thing or a bad thing. So far, 
> however, iirc we have not had any issues sticking with c89 and I don't think 
> the above would warrant such a change. IMO of course.
> 
> I might not have been clear, above. I'm not suggesting changing things for the
> customary build, leave that (at least on httpd 2.4) as -std=c89. I think we 
> should
> have this discussion of when we will begin accepting c99 source patches, but
> that isn't the immediate problem you've tripped over.
> 
> I see several options;
> 
>   Only for maintainer mode, where we are strictly handling all errors, always
>   accept all -std=c99 behaviors (fix any legacy pre-c99 issues that may 
> arise.)
>   All the system headers using c99 (or earlier) semantics should behave well.
> 
>   Or, for maintainer mode, always relax the comments restriction only so we
>   have -std=c89 -Werror -Wall -Wno-error=comment (but not modified in the
>   modules/filters/config.m4 where it isn't apparent who toggled this.) You 
> can 
>   almost call this c99-lite which solves one c99'ism in newer system headers
>   without allowing all the c99'isms in system headers.
> 
>   Or, staying closest to the proposed patch, add -Wno-error=comment only
>   to mod_proxy_html's CPPFLAGS, and stop messing with the rest of the
>   compilation for a single module.
> 
> In every case, I'm expecting we still adhere to c89, especially in httpd-2.4
> branch. A typical compilation (non-maintainer-mode) should catch most
> of those irregularities.

My pov:
- as long as 2.4.x is our only release branch, I'd like trunk maintainer mode 
to stay compatible
- I would like to switch to c99 as soon as 2.6.0 is out
- The CPPFLAGS switch for the module only seems to be the least intrusive

Cheers, Stefan



Re: 2.4.38

2018-11-09 Thread Stefan Eissing
*to thank* (oh my)

> Am 09.11.2018 um 16:51 schrieb Stefan Eissing :
> 
> I would like all who replied in this thread for their feedback. It is good
> to hear that many are looking forward to frequent releases, especially as
> the field we are all working in continues to develop.
> 
> Apache httpd is a server capable of many things, all configurable in various
> ways and even extendable by modules beyond our control here. In short, the
> combinations in which this software is used is beyond our capabilities to
> verify absolutely.
> 
> That makes it tricky, when bringing in "new stuff", not to break anything.
> Because, frankly, before it breaks, we are not always aware that it was used
> that way. (Would we be, we would have tried to fix is before release - 
> ideally).
> 
> So, the chance is high that releases we do will work for most of you.
> AND the chance is high that releases might break something for some of you 
> (hopefully a few).
> 
> We can wiggle the probabilities by a range of activities:
> - more test cases
> - less new features
> but we will never eliminate them. Complexity is too high.
> 
> That is where distros play a crucial role, IMO, as they invest in
> stabilization of the many products they integrate and, as a user,
> you can select from a range of options depending on your willingness
> to bleed vs. desire for stability.
> 
> But people who directly use the product from us are as least as
> important. I got a lot of feedback on HTTP/2 that way (read: you
> found my mistakes) and the quality we have today would not have been
> possible without that. Which benefits everyone. So, thank you!
> 
> Cheers,
> 
> Stefan
> 
>> Am 09.11.2018 um 15:54 schrieb Moradhassel, Kavian :
>> 
>> +1 (as one of the 99.99%)
>> 
>> In particular: "I'd prefer frequent releases and honest changelogs."
>> 
>> 
>> -Original Message-
>> From: Niklas Edmundsson 
>> Reply-To: "dev@httpd.apache.org" 
>> Date: Friday, November 9, 2018 at 8:10 AM
>> To: "dev@httpd.apache.org" 
>> Subject: [**EXTERNAL**] Re: 2.4.38
>> 
>> 
>> I usually don't like top-posts, but I just want to say that I agree 
>> completely with everything Barry stated below.
>> 
>> If you as an admin want an easy life, use the distro version.
>> 
>> If you have good reasons to build yourself simply suck it up and 
>> accept the maintenance pain (which it is, since you need to cater for 
>> updating all the dependencies as well if you want all the latest in 
>> features/fixes). And do read the release notes and upgrade only when 
>> there is a need.
>> 
>> Btw, regular releases increases the chance of distros picking up a 
>> somewhat current version with known fixes when they roll a new distro 
>> release.
>> 
>> I'd prefer frequent releases and honest changelogs. I'm more scared by 
>> projects that have no releases, since that tends to show dead 
>> development or some kind of idealistic view that software can be 
>> "finished" and not needing any more work done on it...
>> 
>>> On Fri, 9 Nov 2018, Barry Pollard wrote:
>>> 
>>> 
>>> 
>>> Disagree. My 2 cents as a watcher, administrator and user:
>>> 
>>> 1/ they have better things to do
>>> 
>>> Then don’t take the release! If a release contains security patches (so 
>>> they should take it), then I don’t see how hiding the issue by holding back 
>>> the release helps.
>>> 
>>> 2/ it gives impression of immature and buggy software - this gives thoughts 
>>> towards alternatives,  IRC shows many admins have no loyalty todays much of 
>>> todays software (well, windows fanbois excepted.
>>> 
>>> Massively disagree. Frequent release to me give the impression of an 
>>> actively maintained and evolving project. And there are a lot of changes in 
>>> the HTTP space (HTTP/2, move to encryption, increased awareness on 
>>> security...etc.).
>>> 
>>> 3/ As a consequence of 1 & 2, they will not upgrade, this might be trivial 
>>> for little thigs, but when a nasty bug comes out, this is what comes to 
>>> mind"  oh fsck it, we just upgraded httpd  last week, screw it, we'll wait" 
>>> - they get bitten, CIOs demand heads, remaining souls dump httpd and 
>>> install nginx or some other alternative
>>> 
>>> Discussed above. And nginx releases monthly (http://nginx.org/en/CHANGES) 
>>> which I’d be happy if Apache HTTPD moved to.
>>> 
>

Re: 2.4.38

2018-11-09 Thread Stefan Eissing
I would like all who replied in this thread for their feedback. It is good
to hear that many are looking forward to frequent releases, especially as
the field we are all working in continues to develop.

Apache httpd is a server capable of many things, all configurable in various
ways and even extendable by modules beyond our control here. In short, the
combinations in which this software is used is beyond our capabilities to
verify absolutely.

That makes it tricky, when bringing in "new stuff", not to break anything.
Because, frankly, before it breaks, we are not always aware that it was used
that way. (Would we be, we would have tried to fix is before release - ideally).

So, the chance is high that releases we do will work for most of you.
AND the chance is high that releases might break something for some of you 
(hopefully a few).

We can wiggle the probabilities by a range of activities:
- more test cases
- less new features
but we will never eliminate them. Complexity is too high.

That is where distros play a crucial role, IMO, as they invest in
stabilization of the many products they integrate and, as a user,
you can select from a range of options depending on your willingness
to bleed vs. desire for stability.

But people who directly use the product from us are as least as
important. I got a lot of feedback on HTTP/2 that way (read: you
found my mistakes) and the quality we have today would not have been
possible without that. Which benefits everyone. So, thank you!

Cheers,

Stefan

> Am 09.11.2018 um 15:54 schrieb Moradhassel, Kavian :
> 
> +1 (as one of the 99.99%)
> 
> In particular: "I'd prefer frequent releases and honest changelogs."
> 
> 
> -Original Message-
> From: Niklas Edmundsson 
> Reply-To: "dev@httpd.apache.org" 
> Date: Friday, November 9, 2018 at 8:10 AM
> To: "dev@httpd.apache.org" 
> Subject: [**EXTERNAL**] Re: 2.4.38
> 
> 
> I usually don't like top-posts, but I just want to say that I agree 
> completely with everything Barry stated below.
> 
> If you as an admin want an easy life, use the distro version.
> 
> If you have good reasons to build yourself simply suck it up and 
> accept the maintenance pain (which it is, since you need to cater for 
> updating all the dependencies as well if you want all the latest in 
> features/fixes). And do read the release notes and upgrade only when 
> there is a need.
> 
> Btw, regular releases increases the chance of distros picking up a 
> somewhat current version with known fixes when they roll a new distro 
> release.
> 
> I'd prefer frequent releases and honest changelogs. I'm more scared by 
> projects that have no releases, since that tends to show dead 
> development or some kind of idealistic view that software can be 
> "finished" and not needing any more work done on it...
> 
> On Fri, 9 Nov 2018, Barry Pollard wrote:
> 
>> 
>> 
>> Disagree. My 2 cents as a watcher, administrator and user:
>> 
>> 1/ they have better things to do
>> 
>> Then don’t take the release! If a release contains security patches (so they 
>> should take it), then I don’t see how hiding the issue by holding back the 
>> release helps.
>> 
>> 2/ it gives impression of immature and buggy software - this gives thoughts 
>> towards alternatives,  IRC shows many admins have no loyalty todays much of 
>> todays software (well, windows fanbois excepted.
>> 
>> Massively disagree. Frequent release to me give the impression of an 
>> actively maintained and evolving project. And there are a lot of changes in 
>> the HTTP space (HTTP/2, move to encryption, increased awareness on 
>> security...etc.).
>> 
>> 3/ As a consequence of 1 & 2, they will not upgrade, this might be trivial 
>> for little thigs, but when a nasty bug comes out, this is what comes to 
>> mind"  oh fsck it, we just upgraded httpd  last week, screw it, we'll wait" 
>> - they get bitten, CIOs demand heads, remaining souls dump httpd and install 
>> nginx or some other alternative
>> 
>> Discussed above. And nginx releases monthly (http://nginx.org/en/CHANGES) 
>> which I’d be happy if Apache HTTPD moved to.
>> 
>> 4/ dont be fooled into thinking  its the package managers role, many 
>> networks run on RedHat EL, SuseEL, and debian, but far from all - and even 
>> those distro package maintainers get sick to F'n death of it after a while 
>> and skip updates.
>> 
>> I do wish Apache would run its own “official” repo to make upgrading to 
>> latest easier. Don’t have the expertise to help with this and understand it 
>> was done in the past and given up due to lack of people who did but still 
>> think it’s a shame we don’t. I think this is an area nginx does stand out. 
>> Upgrading Nginx is often as simple as “yum update” or “apt-get”. They even 
>> run a repo for their mainline version for those that want to be bleeding 
>> edge.
>> 
>> Do not be delusional - this has happened many times before.
>> 
>> I give you dovecot as example, it wasnt that long ago a new release was 

Re: 2.4.38

2018-11-08 Thread Stefan Eissing


> Am 07.11.2018 um 16:15 schrieb Jim Jagielski :
> 
> Now that we have a 2.4.37 out, one in which the number of enhancements and 
> fixes and feature were limited, it makes sense to consider having a 2.4.38 
> release somewhat "soonish", esp considering the number of backports that lack 
> only a single vote.
> 
> Comments?

I am all for it. And I have reasons! :-)

1. We are well on track to automate the whole thing nicely. We need some more 
iterations to make it tight. This is easier done in short intervals.
2. I think regular releases are good for the people here and motivate people 
from outside to contribute.
3. h2 and Let's Encrypt still get attention and there are some small 
improvements I'd like to throw out there. Nothing serious, just would be nice.

Cheers,

Stefan

Re: Load balancing and load determination

2018-11-05 Thread Stefan Eissing


> Am 05.11.2018 um 16:58 schrieb William A Rowe Jr :
> 
> On Mon, Nov 5, 2018 at 7:48 AM jean-frederic clere  wrote:
> On 30/10/2018 13:53, Jim Jagielski wrote:
> > As some of you know, one of my passions and area of focus is
> > on the use of Apache httpd as a reverse proxy and, as such, load
> > balancing, failover, etc are of vital interest to me.
> > 
> > One topic which I have mulling over, off and on, has been the
> > idea of some sort of universal load number, that could be used
> > and agreed upon by web servers. Right now, the reverse proxy
> > "guesses" the load on the backend servers which is OK, and
> > works well enough, but it would be great if it actually "knew"
> > the current loads on those servers. I already have code that
> > shares basic architectural info, such as number of CPUs, available
> > memory, loadavg, etc which can help, of course, but again, all
> > this info can be used to *infer* the current status of those backend
> > servers; it doesn't really provide what the current load actually
> > *is*.
> > 
> > So I was thinking maybe some sort of small, simple and "fast"
> > benchmark which could be run by the backends as part of their
> > "status" update to the front-end reverse proxy server... something
> > that shows general capability at that point in time, like Hanoi or
> > something similar. Or maybe some hash function. Some simple code
> > that could be used to create that "universal" load number.
> > 
> > Thoughts? Ideas? Comments? Suggestions? :)
> 
> having the back-ends to provide the load they are able to handle
> lbfactor (via w_lf or somethere similar. That requires the back-ends to
> be able to send request to httpd balancer-manager handler.
> 
> Not really. I'd suggest a response header, travelling with each response
> back to the balancer, which can be composed quickly enough to share
> a play-by-play snapshot of the availability of that backend. This adds
> next to no traffic and minimal cpu drain if composed cleanly. And it can
> optionally be axed by the balancer in the response to the client.
> 
> The last thing we want are the routing headaches of contacting an
> ever-changing list one-or-many potential balancers. And we can't
> rely on a dying lbmember to "check in" that it isn't functional. Since
> the balancer must already start requests to the backend, having that
> backend supplement the responses with its health status is simple.

Funnily enough, I did my master thesis (is that a word?) a long, long
while ago on scheduling in distributed systems. And with "distributed"
the general tricky thing is that there is not global knowledge of the
system state.

While any load indicator reported from the backends might look very
useful, once you deal with several front ends, this degenerates quickly
(where each frontend makes its own decision without talking to each
other).

If you detect and exclude any failing backends (heartbeat), then, with
growing number of back- and frontends, it's very hard to beat a random
job distribution.

I found that, in general, pulling works slightly better than pushing. The
scenario here would be that backends ask frontends for requests to execute.
That is also very stable in case of backend failures, of course.

tl;dr

If your problem scenario includes more than a single frontend, go for random.

Cheers,

Stefan



Re: dual port 80 443

2018-10-26 Thread Stefan Eissing


> Am 26.10.2018 um 08:48 schrieb Edwardo Garcia :
> 
> Hi,
> We have only few domains to manage, usually either http or https, but we have 
> lately had requests for both (we  know defeat purpose but customer knows what 
> they want and they no take monetary or personal informations on website)
> 
> I know this works with duplication of virtualhosts, but should it also work 
> with
> 
> ...
> 
> To avoid duplicating? 
> nginx does not seem to have this limitation, so I'm surprised httpd2 does.
> 
> If I omit ports, it will errors on http  if ssl engine on.
> 
> or have I overlooked option?

The usual approach is, I think, to put the generic config into its own file and 
include that in each vhost. It's not ideal.

Cheers,

Stefan



Re: Test framework regressions - spelling and usertrack

2018-10-22 Thread Stefan Eissing
dAMn! uSABilitY StRiks aGAIn!

> Am 22.10.2018 um 17:34 schrieb Jim Jagielski :
> 
> OK, I think I know what may be happening; I am guessing its due to the macOS 
> file system being case insensitive but case preserving...



Re: t/modules/http2.t: Run only if OpenSSL >= 1.0.0 is available

2018-10-22 Thread Stefan Eissing
Thanks a lot!

> Am 22.10.2018 um 14:06 schrieb Rainer Jung :
> 
> This seems to work nicely, committed in r1844546. Tests with old OpenSSL 
> either in client or server result in TLSv1 and disable h2 tests. TLS test 
> requests that result in TLSv1_2 or TLSv1_3 enable h2 tests.
> 
> Regards,
> 
> Rainer
> 
> Am 22.10.2018 um 12:37 schrieb Rainer Jung:
>> I wonder whether it would be easier to check whether a TLS connection uses 
>> TLS 1.2 or later and disable the h2 test if not.
>> Nevertheless the module for checking the server version might be useful, but 
>> here I guess checking the TLS version is more appropriate.
>> But that might mean, that the test runs with OpenSSL 0.9.8zh in the client. 
>> At least I see some TLSv1.2 reuests during the test suite run even when 
>> using 0.9.8zh in the client. It ever happens with that version in the server.
>> Will look into it.
>> Regards,
>> Rainer
>> Am 21.10.2018 um 14:28 schrieb Daniel Ruggeri:
>>> 
>>> On 10/21/2018 6:46 AM, Rainer Jung wrote:
>>>> Am 18.10.2018 um 14:23 schrieb Stefan Eissing:
>>>>>> Am 18.10.2018 um 14:12 schrieb Rainer Jung :
>>>>>> 
>>>>>> - t/modules/http2.t fails when the server is build using OpenSSL
>>>>>> 0.9.8zh with the "Bad plan.  You planned 52 tests..." message
>>>>>> indicating, that h2 using TLS does not work. It happens on all
>>>>>> platforms, but not if the client also uses OpenSSL 0.9.8zh.
>>>>>> 
>>>>>> I don't know whether that is expected for old OpenSSL, so can not
>>>>>> judge on criticality.
>>>>> 
>>>>> AFAICT, correct me if I am wrong, OpenSSL 0.9.8 does not support
>>>>> TLSv1.2 and is therefore unusable with h2. The test suite seems to be
>>>>> unprepared for this scenario. I will remove it after the next
>>>>> release. It is not worth fixing in its current form.
>>>> 
>>>> I added a check agains the test suite OpenSSL version in r1844483.
>>>> 
>>>> I have an aditional check for the server version available.
>>>> Unfortunately I didn't find a really easy way, so here's a small
>>>> module that one can query
>>>> (c-modules/test_ssl_version/mod_test_ssl_version.c), mostly a
>>>> shortened form of mod_test_ssl.c:
>>>> 
>>>>  SNIP =
>>>> #define HTTPD_TEST_REQUIRE_APACHE 2
>>>> 
>>>> #if CONFIG_FOR_HTTPD_TEST
>>>> 
>>>> 
>>>>  
>>>>  SetHandler test-ssl-version-lookup
>>>>  
>>>> 
>>>> 
>>>> #endif
>>>> 
>>>> #include "httpd.h"
>>>> #include "http_config.h"
>>>> #include "http_protocol.h"
>>>> #include "http_log.h"
>>>> #include "ap_config.h"
>>>> #include "apr_optional.h"
>>>> 
>>>> #if AP_MODULE_MAGIC_AT_LEAST(20040425, 0) /* simply include mod_ssl.h
>>>> if using >= 2.1.0 */
>>>> 
>>>> #include "mod_ssl.h"
>>>> 
>>>> #else
>>>> /* For use of < 2.0.x, inline the declaration: */
>>>> 
>>>> APR_DECLARE_OPTIONAL_FN(char *, ssl_var_lookup,
>>>>  (apr_pool_t *, server_rec *,
>>>>   conn_rec *, request_rec *,
>>>>   char *));
>>>> 
>>>> #endif
>>>> 
>>>> static APR_OPTIONAL_FN_TYPE(ssl_var_lookup) *var_lookup;
>>>> 
>>>> static void import_ssl_var_lookup(void)
>>>> {
>>>>  var_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
>>>> }
>>>> 
>>>> static int test_ssl_version_lookup(request_rec *r)
>>>> {
>>>>  char *value;
>>>> 
>>>>  if (strcmp(r->handler, "test-ssl-version-lookup")) {
>>>>  return DECLINED;
>>>>  }
>>>> 
>>>>  if (r->method_number != M_GET) {
>>>>  return DECLINED;
>>>>  }
>>>> 
>>>>  if (!var_lookup) {
>>>>  ap_rputs("ssl_var_lookup is not available", r);
>>>>  return OK;
>>>>  }
>>>> 
>>>>  value = var_lookup(r->pool, r->server,
>>>&g

Re: [VOTE] Release httpd-2.4.37

2018-10-19 Thread Stefan Eissing
+1 

MacOS 10.14, OpenSSL 1.0.2p + 1.1.1, mod-h2 suite, mod-md suite
Ubuntu 18.04 LTS, OpenSSL 1.1.0h, mod-h2 suite

Thanks for RM'ing, Daniel!

> Am 18.10.2018 um 16:36 schrieb Daniel Ruggeri :
> 
> Hi, all;
>   Please find below the proposed release tarball and signatures:
> https://dist.apache.org/repos/dist/dev/httpd/
> 
> I would like to call a VOTE over the next few days to release this candidate 
> tarball as 2.4.37:
> [ ] +1: It's not just good, it's good enough!
> [ ] +0: Let's have a talk.
> [ ] -1: There's trouble in paradise. Here's what's wrong.
> 
> The computed digests of the tarball up for vote are:
> sha1: b0521606d1df54bb425adcdecf6348f126aa352c *httpd-2.4.37.tar.gz
> sha256: aa97a834a32d51974be8d8a013b561e28d327387cb1da2c3c2762acd0146aabd 
> *httpd-2.4.37.tar.gz
> 
> -- 
> Daniel Ruggeri



Re: [NOTICE] Intent to T 2.4.37 - about 12:00 GMT tomorrow

2018-10-18 Thread Stefan Eissing



> Am 18.10.2018 um 14:12 schrieb Rainer Jung :
> 
> - t/modules/http2.t fails when the server is build using OpenSSL 0.9.8zh with 
> the "Bad plan.  You planned 52 tests..." message indicating, that h2 using 
> TLS does not work. It happens on all platforms, but not if the client also 
> uses OpenSSL 0.9.8zh.
> 
> I don't know whether that is expected for old OpenSSL, so can not judge on 
> criticality.

AFAICT, correct me if I am wrong, OpenSSL 0.9.8 does not support TLSv1.2 and is 
therefore unusable with h2. The test suite seems to be unprepared for this 
scenario. I will remove it after the next release. It is not worth fixing in 
its current form.

-Stefan



Re: svn commit: r1844002 - in /httpd/httpd/trunk: CHANGES modules/ssl/ssl_engine_config.c

2018-10-18 Thread Stefan Eissing
Ok, the vote storm (category 3) was released and my proposal is moot. ;-)

> Am 18.10.2018 um 11:26 schrieb Stefan Eissing :
> 
> Can we not just make a ssl-for-2.4.37 branch, merge the mod_ssl related 
> changes there and do one row of tests and vote on it? Maybe attach the branch 
> revision to the vote that was tested...
> 
> Seems to be able to save work, or?
> 
>> Am 18.10.2018 um 11:22 schrieb Yann Ylavic :
>> 
>> On Thu, Oct 18, 2018 at 11:18 AM Rainer Jung  wrote:
>>> 
>>> This fix at least formally applies to 2.4.x as well? Shouldn't it get
>>> backported?
>> 
>> +1
>> 
>> Regards,
>> Yann.
> 



Re: svn commit: r1844002 - in /httpd/httpd/trunk: CHANGES modules/ssl/ssl_engine_config.c

2018-10-18 Thread Stefan Eissing
Can we not just make a ssl-for-2.4.37 branch, merge the mod_ssl related changes 
there and do one row of tests and vote on it? Maybe attach the branch revision 
to the vote that was tested...

Seems to be able to save work, or?

> Am 18.10.2018 um 11:22 schrieb Yann Ylavic :
> 
> On Thu, Oct 18, 2018 at 11:18 AM Rainer Jung  wrote:
>> 
>> This fix at least formally applies to 2.4.x as well? Shouldn't it get
>> backported?
> 
> +1
> 
> Regards,
> Yann.



Re: [NOTICE] Intent to T 2.4.37 - about 12:00 GMT tomorrow

2018-10-17 Thread Stefan Eissing



> Am 17.10.2018 um 13:43 schrieb Graham Leggett :
> 
> On 17 Oct 2018, at 13:41, Daniel Ruggeri  wrote:
> 
>> Hi, all;
>> With the fix for detected OpenSSL 1.1.1 issues now backported to 2.4.x, I 
>> would like to tag the next version of our venerable server soon.
>> 
>> I have already successfully completed the test suite against my "latest 
>> sources" docker environment and am watching for any smoke detected in [1]. 
>> Feeling good about this one :-)
>> 
>> How about roughly 24 hours from now?
> 
> +1.
> 
> I see a flurry of openssl 1.1.1 / TLS1.3 related activity on various other 
> tools out there (sendmail, etc), so we’re in good company.
> 

+1.

Cheers, Stefan

Re: mod_headers best practices and headers duplicated in the response

2018-10-17 Thread Stefan Eissing



> Am 15.10.2018 um 13:45 schrieb Luca Toscano :
> 
> Hi everybody,
> 
> apologies if this subject has been brought up in the past but I didn't
> find much. I have been working on some bugs like
> https://bz.apache.org/bugzilla/show_bug.cgi?id=62380 in which users
> report responses with the same header duplicated. To keep the story
> short, it seems that everything is due to the fact that sometimes
> httpd fills both r->headers_out and r->err_headers_out with the same
> header, and both get added to the response.
> 
> Example from 62380:
> 
> - Simple ProxyPass configuration with mod_proxy_http, the (HTTP)
> backend returns a response with the X-Foo header set to baz
> - Header always set X-Foo "bar" (in the httpd.conf)
> 
> This leads to the following in the response:
> [..]
> X-Foo: baz
> [..]
> X-Foo: bar
> [..]
> 
> This happens because, afaict, mod_proxy_http returns the backend
> headers into r->headers_out, while Header always set adds it to
> r->err_headers_out. With mod_proxy_fcgi the behavior is different
> since both headers end up in r->err_headers_out (so only X-Foo: bar is
> present in the response).
> I tried to come up with some patch, but the more I think about it the
> less I am inclined to change the current behavior, but only to improve
> the docs to be more explicit about the two header lists:
> 
> Header onsuccess verb [..]  -> works on r->headers_out
> Header always verb [..] -> works on r->err_headers_out
> 
> So in the above baz/bar example, if the users really wants to be sure
> that an override happens, it should:
> 
> Header onsuccess unset X-Foo  (onsuccess can be omitted in here, but
> it seems clearer to be explicit)
> Header always set X-Foo bar
> 
> Same thing for all the other verbs of course (setifempty, merge,
> etc..). It is also kinda explained in
> https://httpd.apache.org/docs/2.4/mod/mod_headers.html#header but in
> my opinion not super clearly, it is easy to get confused (like I was)
> while reading it the first 3/4 times.
> 
> Anything that I am missing or this is "only" a documentation bug fix?
> (Also something worth to change to be more user friendly in 2.6 tbh).

Not an expert on mod_headers, but its documentation mentions "add" and "set"
with different semantics. From a user's point of view, it seems that "set"
should replace any pre-existing header(s) of the same name.

Now, this might be difficult or even impossible to fix in 2.4.x without
breakage somewhere else. Documenting existing behaviour, offering workaround
as you describe, looks good to me.

Ideally, any header manipulation would work on a _normalized_ header
dictionary, so that operations and their outcome are more clear. 

The HTTP header _serialization_ is quite flexible so that values can be added 
later
in processing. But this then can make things complicated. Especially since
some header field names are unlike others...

-Stefan

Re: h2 broken in 2.4.36 with OpenSSL 1.1.1? Related to SSL_MODE_AUTO_RETRY?

2018-10-16 Thread Stefan Eissing
> Am 15.10.2018 um 23:16 schrieb Rainer Jung :
> 
> Adjusted SSL_read() rc value 0 handling applied in r1843954 to trunk. I'll 
> let it sit there until tomorrow for comments and then suggest for backport.

Thanks, Rainer! 

The h2 test suite runs as smoothly as it did before on my machine with OpenSSL 
1.1.1 in trunk. I am all for backporting and testing von 2.4.x on this.

-Stefan


> 
> Am 15.10.2018 um 12:55 schrieb Rainer Jung:
>> Am 15.10.2018 um 10:02 schrieb Stefan Eissing:
>>> 
>>> 
>>>> Am 14.10.2018 um 00:46 schrieb Rainer Jung :
>>>> 
>>>> It seems the h2 failure only happens when building httpd against OpenSSL 
>>>> 1.1.1 (independent of TLS version used). I did a quick check with an httpd 
>>>> build against 1.1.0i and there the same vhost of the test framework 
>>>> instance worked with the same clients, that failed for 1.1.1.
>>>> 
>>>> The client side OpenSSL version seems not to matter.
>>>> 
>>>> When using 1.1.1 in the server even browsers seem to fail with h2.
>>>> 
>>>> Anyone who can successfully use h2 with 2.4.36 build against OpenSSL 1.1.1?
>>> 
>>> Me, and I got reports from others.
>>> 
>>>> When comparing trace logs between the 1.1.1 server and the 1.1.0i server, 
>>>> one can see, that a failing OpenSSL read (0 bytes) results in APR_EOF 
>>>> (70014) for 1.1.1 but in errno 11 (EAGAIN) for 1.1.0i. I wonder whether 
>>>> this is due to the new SSL_MODE_AUTO_RETRY and maybe the following change:
>>>> 
>>>> +#if OPENSSL_VERSION_NUMBER >= 0x1010100fL
>>>> +/* For OpenSSL >=1.1.1, disable auto-retry mode so it's possible
>>>> + * to consume handshake records without blocking for app-data.
>>>> + * https://github.com/openssl/openssl/issues/7178 */
>>>> +SSL_CTX_clear_mode(ctx, SSL_MODE_AUTO_RETRY);
>>>> +#endif
>>> 
>>> Hmm, just read the ticket made by Joe regarding this change.
>>> 
>>> I try to summarize my understanding:
>>> 
>>> - this has nothing to do with HTTP/2 in particular. It's only triggered by 
>>> the h2 calling sequence.
>>> - SSL_read() only returns transport data, but sometimes reads (part of) TLS 
>>> meta data, such as handshake messages.
>>> - When it reads this meta data (and has processed it), it can either do 
>>> another read directly after or return with
>>>its equivalent of EAGAIN. By default, it does the first.
>>> - The change in handshake handling between TLSv1.2 and TLSv1.3 led, in 
>>> Joe's testing, to the case where SSL_read()'s
>>>internal re-read() on the socket blocked, as the client was not sending 
>>> anything.
>>> - So, after discussion on the openssl issue tracker, Joe changed the 
>>> OpenSSL behaviour of this by inserting the code above.
>>> - Now, when SSL_read() is called, processes some meta data, it will not 
>>> read() on the socket again, but return a read of length 0.
>>> - mod_ssl interprets this as EOF:
>>> ssl_engine_io.c, lines 673..
>>> 
>>>  else if (rc == 0) {
>>>  /* If EAGAIN, we will loop given a blocking read,
>>>   * otherwise consider ourselves at EOF.
>>>   */
>>> 
>>> and returns APR_EOF to h2 which then shuts the connection down.
>>> 
>>> To my understanding, mod_ssl's ssl_engine_io.c, ssl_io_input_read() has the 
>>> special case handling that:
>>> - if it gets a SSL_ERROR_WANT_READ from SSL_read(), it remembers that, 
>>> calls again
>>> - if SSL_read() == 0, non-blocking, it
>>>a) on having seen SSL_ERROR_WANT_READ before, return APR_EGAIN
>>>b) otherwise, return APR_EOF
>>> 
>>> OpenSSL's documentation of SSL_read() now states:
>>> 
>>>> Old documentation indicated a difference between 0 and -1, and that -1 was 
>>>> retry-able.
>>>> You should instead call SSL_get_error() to find out if it's retry-able.
>>> 
>>> So, we should change our logic here. Anyone having the courage to do so? ;-)
>> Thanks for the further investigations.
>> I'm currently testing the following patch which looks OK wrt. test suite 
>> results. Need to run more combinations (OpenSSL version client versus 
>> server) though. Server with 1.1.1 and with 1.0.2p both look OK (including 
>> the h2 tests). Maybe some cases could be folded together or be dropped, but 
>> I tried to make t

Re: [VOTE] Release httpd-2.4.36

2018-10-15 Thread Stefan Eissing



> Am 15.10.2018 um 16:11 schrieb Jim Jagielski :
> 
> It's up to the RM on whether or not to release... one can't veto a release 
> and a -1 is not a veto.

Huh? I was referring to "TLS 1.3 support isn't quite yet tested enough to 
warrant a public release". I wanted to point out that without attempting a 
public release, we may not have found this bug for months. I am -1 on 2.4.36 as 
well, in case that was not clear. Don't know how this "veto" came into this...

-Stefan

>> On Oct 15, 2018, at 10:07 AM, Stefan Eissing  
>> wrote:
>> 
>> 
>> 
>>> Am 15.10.2018 um 15:58 schrieb Jim Jagielski :
>>> 
>>> Considering all this, I am changing my vote from a +1 to a -1. I was not 
>>> able to trigger this error, but this shows, at least IMO, that TLS 1.3 
>>> support isn't quite yet tested enough to warrant a public release, unless 
>>> we are super clear that it is "experimental" or "early access"...
>> 
>> I do not see it this black/white way. 
>> 
>> We have found no regression with any SSL != OpenSSL 1.1.1. 
>> We have not even found a bug with TLSv1.3 as such. What we have found is a 
>> behaviour change in OpenSSL where our code relied on either changed or not 
>> well documented behaviour. 
>> 
>> We do not want to ship a version of httpd which has severe interop problems 
>> with the released openssl 1.1.1. 
>> HOWEVER: it is unclear, if this will not also trigger in some scenario when 
>> one links 2.4.35 with OpenSSL 1.1.1.
>> 
>> I am all in favor of pushing a 2.4.37 immediately after this bug is fixed. 
>> We will not solve any remaining problems by letting it stew in the 
>> repository. 
>> 
>> -Stefan
>> 
>>> 
>>>> On Oct 15, 2018, at 4:06 AM, Stefan Eissing  
>>>> wrote:
>>>> 
>>>> 
>>>> 
>>>>> Am 14.10.2018 um 23:46 schrieb Daniel Ruggeri :
>>>>> 
>>>>> Hi, Helmut;
>>>>> Note that the vote may run longer than 72 hours as 72 is the minimum. As 
>>>>> it stands now, we have more than 3 binding +1 votes, but I am waiting for 
>>>>> closure on the conversation on-list about the tests with reported H2/TLS 
>>>>> 1.3 failures. Since this is one of the primary features of this release, 
>>>>> I want to be sure the topic gets due attention.
>>>> 
>>>> See my mail on the other thread. It seems that h2 traffic triggers a call 
>>>> sequence that exposes a change in OpenSSL behaviour of SSL_read() between 
>>>> 1.1.0 and 1.1.1. It looks as if mod_ssl interpreted the return codes of 
>>>> SSL_read() in a way that no longer works and that we need to change 
>>>> mod_ssl handling here.
>>>> 
>>>> Waiting on confirmation  or rebuttal of my analysis on the other thread.
>>>> 
>>>> Cheers,
>>>> 
>>>> Stefan
>>>> 
>>>>> -- 
>>>>> Daniel Ruggeri
>>>>> 
>>>>> On October 14, 2018 4:44:04 PM CDT, "Helmut K. C. Tessarek" 
>>>>>  wrote:
>>>>> On 2018-10-10 15:18, Daniel Ruggeri wrote:
>>>>> Hi, all;
>>>>> Please find below the proposed release tarball and signatures:
>>>>> https://dist.apache.org/repos/dist/dev/httpd/
>>>>> 
>>>>> I would like to call a VOTE over the next few days to release this
>>>>> candidate tarball as 2.4.36:
>>>>> [ ] +1: It's not just good, it's good enough!
>>>>> [ ] +0: Let's have a talk.
>>>>> [ ] -1: There's trouble in paradise. Here's what's wrong.
>>>>> 
>>>>> The computed digests of the tarball up for vote are:
>>>>> sha1: e40e7a879b84df860215b8a80f2a535534a1c4b4 *httpd-2.4.36.tar.gz
>>>>> sha256: ef788fb7c814acb2506a8b758a1a3f91f368f97bd4e6db16e98001f468e8e288
>>>>> *httpd-2.4.36.tar.gz
>>>>> 
>>>>> 72h have passed, so what is the outcome of the vote?
>>>>> 
>>>> 
>>> 
>> 
> 



Re: [VOTE] Release httpd-2.4.36

2018-10-15 Thread Stefan Eissing



> Am 15.10.2018 um 15:58 schrieb Jim Jagielski :
> 
> Considering all this, I am changing my vote from a +1 to a -1. I was not able 
> to trigger this error, but this shows, at least IMO, that TLS 1.3 support 
> isn't quite yet tested enough to warrant a public release, unless we are 
> super clear that it is "experimental" or "early access"...

I do not see it this black/white way. 

We have found no regression with any SSL != OpenSSL 1.1.1. 
We have not even found a bug with TLSv1.3 as such. What we have found is a 
behaviour change in OpenSSL where our code relied on either changed or not well 
documented behaviour. 

We do not want to ship a version of httpd which has severe interop problems 
with the released openssl 1.1.1. 
HOWEVER: it is unclear, if this will not also trigger in some scenario when one 
links 2.4.35 with OpenSSL 1.1.1.

I am all in favor of pushing a 2.4.37 immediately after this bug is fixed. We 
will not solve any remaining problems by letting it stew in the repository. 

-Stefan

> 
>> On Oct 15, 2018, at 4:06 AM, Stefan Eissing  
>> wrote:
>> 
>> 
>> 
>>> Am 14.10.2018 um 23:46 schrieb Daniel Ruggeri :
>>> 
>>> Hi, Helmut;
>>> Note that the vote may run longer than 72 hours as 72 is the minimum. As it 
>>> stands now, we have more than 3 binding +1 votes, but I am waiting for 
>>> closure on the conversation on-list about the tests with reported H2/TLS 
>>> 1.3 failures. Since this is one of the primary features of this release, I 
>>> want to be sure the topic gets due attention.
>> 
>> See my mail on the other thread. It seems that h2 traffic triggers a call 
>> sequence that exposes a change in OpenSSL behaviour of SSL_read() between 
>> 1.1.0 and 1.1.1. It looks as if mod_ssl interpreted the return codes of 
>> SSL_read() in a way that no longer works and that we need to change mod_ssl 
>> handling here.
>> 
>> Waiting on confirmation  or rebuttal of my analysis on the other thread.
>> 
>> Cheers,
>> 
>> Stefan
>> 
>>> -- 
>>> Daniel Ruggeri
>>> 
>>> On October 14, 2018 4:44:04 PM CDT, "Helmut K. C. Tessarek" 
>>>  wrote:
>>> On 2018-10-10 15:18, Daniel Ruggeri wrote:
>>> Hi, all;
>>>  Please find below the proposed release tarball and signatures:
>>> https://dist.apache.org/repos/dist/dev/httpd/
>>> 
>>> I would like to call a VOTE over the next few days to release this
>>> candidate tarball as 2.4.36:
>>> [ ] +1: It's not just good, it's good enough!
>>> [ ] +0: Let's have a talk.
>>> [ ] -1: There's trouble in paradise. Here's what's wrong.
>>> 
>>> The computed digests of the tarball up for vote are:
>>> sha1: e40e7a879b84df860215b8a80f2a535534a1c4b4 *httpd-2.4.36.tar.gz
>>> sha256: ef788fb7c814acb2506a8b758a1a3f91f368f97bd4e6db16e98001f468e8e288
>>> *httpd-2.4.36.tar.gz
>>> 
>>> 72h have passed, so what is the outcome of the vote?
>>> 
>> 
> 



Re: [VOTE] Release httpd-2.4.36

2018-10-15 Thread Stefan Eissing



> Am 15.10.2018 um 15:51 schrieb William A Rowe Jr :
> 
> 
> 
> On Mon, Oct 15, 2018 at 3:06 AM Stefan Eissing  
> wrote:
> 
> See my mail on the other thread. It seems that h2 traffic triggers a call 
> sequence that exposes a change in OpenSSL behaviour of SSL_read() between 
> 1.1.0 and 1.1.1. It looks as if mod_ssl interpreted the return codes of 
> SSL_read() in a way that no longer works and that we need to change mod_ssl 
> handling here.
> 
> Stefan, thanks for the detailed analysis else-thread, and thank you Rainer 
> for the detailed defect report. It would be interesting to trigger this 
> deliberately in the test framework.
>  
> > On October 14, 2018 4:44:04 PM CDT, "Helmut K. C. Tessarek" 
> >  wrote:
> > On 2018-10-10 15:18, Daniel Ruggeri wrote:
> > Hi, all;
> >Please find below the proposed release tarball and signatures:
> > https://dist.apache.org/repos/dist/dev/httpd/
> > 
> > I would like to call a VOTE over the next few days to release this
> > candidate tarball as 2.4.36:
> > [ ] +1: It's not just good, it's good enough!
> > [ ] +0: Let's have a talk.
> > [ ] -1: There's trouble in paradise. Here's what's wrong.
> 
> Based on the observed change of SSL_read which we had not entirely accounted 
> for, I'm -1 for GA release.
> 
> I don't think it's helpful for us to ship this defect in any alpha or beta of 
> trunk. I'd consider it a showstopper.

Agreed.



Re: [VOTE] Release httpd-2.4.36

2018-10-15 Thread Stefan Eissing



> Am 14.10.2018 um 23:46 schrieb Daniel Ruggeri :
> 
> Hi, Helmut;
> Note that the vote may run longer than 72 hours as 72 is the minimum. As it 
> stands now, we have more than 3 binding +1 votes, but I am waiting for 
> closure on the conversation on-list about the tests with reported H2/TLS 1.3 
> failures. Since this is one of the primary features of this release, I want 
> to be sure the topic gets due attention.

See my mail on the other thread. It seems that h2 traffic triggers a call 
sequence that exposes a change in OpenSSL behaviour of SSL_read() between 1.1.0 
and 1.1.1. It looks as if mod_ssl interpreted the return codes of SSL_read() in 
a way that no longer works and that we need to change mod_ssl handling here.

Waiting on confirmation  or rebuttal of my analysis on the other thread.

Cheers,

Stefan

> -- 
> Daniel Ruggeri
> 
> On October 14, 2018 4:44:04 PM CDT, "Helmut K. C. Tessarek" 
>  wrote:
> On 2018-10-10 15:18, Daniel Ruggeri wrote:
> Hi, all;
>Please find below the proposed release tarball and signatures:
> https://dist.apache.org/repos/dist/dev/httpd/
> 
> I would like to call a VOTE over the next few days to release this
> candidate tarball as 2.4.36:
> [ ] +1: It's not just good, it's good enough!
> [ ] +0: Let's have a talk.
> [ ] -1: There's trouble in paradise. Here's what's wrong.
> 
> The computed digests of the tarball up for vote are:
> sha1: e40e7a879b84df860215b8a80f2a535534a1c4b4 *httpd-2.4.36.tar.gz
> sha256: ef788fb7c814acb2506a8b758a1a3f91f368f97bd4e6db16e98001f468e8e288
> *httpd-2.4.36.tar.gz
> 
> 72h have passed, so what is the outcome of the vote?
> 



Re: h2 broken in 2.4.36 with OpenSSL 1.1.1? Related to SSL_MODE_AUTO_RETRY?

2018-10-15 Thread Stefan Eissing
(722): h2_task(81-1): adding request filters
> > 23:39:55.716598 SLAVE h2_h2.c(722): h2_task(66-1): adding request filters
> 
> < 23:35:16.022782 SLAVE ssl_engine_kernel.c(383): AH02034: Subsequent (No.2) 
> HTTPS request received for child 347892350977 (server localhost:8557)
> < 23:35:16.022799 SLAVE h2_task.c(676): h2_task(81-1): start process_request
> < 23:35:16.022811 [http:trace4] SLAVE http_request.c(437): Headers received 
> from client:
> < 23:35:16.022817 [http:trace4] SLAVE http_request.c(441):   User-Agent: 
> curl/7.61.1
> < 23:35:16.022822 [http:trace4] SLAVE http_request.c(441):   Accept: */*
> > 23:39:55.716635 SLAVE ssl_engine_kernel.c(383): AH02034: Subsequent (No.2) 
> > HTTPS request received for child 283467841537 (server localhost:8557)
> > 23:39:55.716646 SLAVE h2_task.c(676): h2_task(66-1): start process_request
> > 23:39:55.716652 [http:trace4] SLAVE http_request.c(437): Headers received 
> > from client:
> > 23:39:55.716658 [http:trace4] SLAVE http_request.c(441): User-Agent: 
> > curl/7.61.1
> > 23:39:55.716661 [http:trace4] SLAVE http_request.c(441):   Accept: */*
> > 23:39:55.716665 [http:trace4] SLAVE http_request.c(441):   Host: 
> > localhost:8557
> 
> But then failure (empty result) for 1.1.1 due to GOAWAY
> 
> < 23:35:16.022824 core_filters.c(525): will flush because of FLUSH bucket
> < 23:35:16.022825 [http:trace4] SLAVE http_request.c(441):   Host: 
> localhost:8557
> < 23:35:16.022828 core_filters.c(535): seen in brigade so far: bytes: 39, 
> non-file bytes: 39, eor buckets: 0, morphing buckets: 0
> < 23:35:16.022833 core_filters.c(554): flushing now
> < 23:35:16.022874 core_filters.c(569): total bytes written: 3220
> < 23:35:16.022882 core_filters.c(580): brigade contains: bytes: 0, non-file 
> bytes: 0, eor buckets: 0, morphing buckets: 0
> < 23:35:16.022887  h2_session.c(715): AH03069: h2_session(81,BUSY,1): sent 
> GOAWAY, err=0, msg=
> < 23:35:16.022895  h2_session.c(1704): AH03078: h2_session(81,DONE,1): 
> transit [BUSY] -- local goaway --> [DONE]
> < 23:35:16.022900 h2_mplx.c(1240): h2_mplx(81): dispatch events
> < 23:35:16.022905 h2_session.c(2364): h2_session(81,DONE,1): process returns
> < 23:35:16.022909  h2_conn.c(217): (70014)End of file found: AH03045: 
> h2_session(81,DONE,1): process, closing conn
> < 23:35:16.022916 h2_session.c(2382): h2_session(81,DONE,1): pre_close
> < 23:35:16.022920 h2_session.c(725): h2_session(81,DONE,1): pool_cleanup
> < 23:35:16.022924  h2_session.c(1704): AH03078: h2_session(81,CLEANUP,1): 
> transit [DONE] -- pre_close --> [CLEANUP]
> < 23:35:16.022929 h2_stream.c(612): h2_stream(81-1,HALF_CLOSED_REMOTE): 
> reset, error=0
> < 23:35:16.022941 h2_stream.c(344): h2_stream(81-1,HALF_CLOSED_REMOTE): 
> dispatch event 2
> < 23:35:16.022945 h2_stream.c(302): h2_stream(81-1,HALF_CLOSED_REMOTE): 
> transit to [CLOSED]
> < 23:35:16.022949 h2_stream.c(211): h2_stream(81-1,CLOSED): closing input
> < 23:35:16.022953 h2_stream.c(344): h2_stream(81-1,CLOSED): dispatch event 3
> < 23:35:16.022957 h2_stream.c(302): h2_stream(81-1,CLOSED): transit to 
> [CLEANUP]
> < 23:35:16.022963 h2_ngn_shed.c(144): AH03394: h2_ngn_shed(81): abort
> 
> Whereas 1.1.0i proceeds processing and sends the result back
> ...
> 
> 
> Regards,
> 
> Rainer
> 
> Am 13.10.2018 um 23:07 schrieb Rainer Jung:
>> Hi Stefan,
>> it is the "input gone" (APR_EOF) case which went unnoticed by me. Although I 
>> patch the test suite to run with trace8 log level, http2 was set to debug in 
>> the test suite config and the "input gone" message is a trace message. See 
>> below for more details. The question is still whether this should have been 
>> handled non-fatally. Currently curl fails to do h2 with httpd 2.4.36 as set 
>> up by the test suite.
>> Am 13.10.2018 um 18:53 schrieb Stefan Eissing:
>>> Hi Rainer,
>>> 
>>> according to the log, the h2 code must be in the H2_SESSION_ST_BUSY state 
>>> and the only cause I see is the same as you, namely an unexpected status 
>>> from h2_session_read(), which should come via
>>> 
>>>  status = ap_get_brigade(c->input_filters,
>>>  session->bbtmp, AP_MODE_READBYTES,
>>>  block? APR_BLOCK_READ : APR_NONBLOCK_READ,
>>>  H2MAX(APR_BUCKET_BUFF_SIZE, readlen));
>>> 
>>> block is 0 here and readlen should be (unless reconfigured via 
>>> H2StreamMaxMemSize) 32kb.
>>> 
>>> Maybe you could just add a log line there to see wha

Re: Failing http2.t in 2.4.36 [Was: NOTICE: Intent to T 2.4.36]

2018-10-13 Thread Stefan Eissing
Hi Rainer,

according to the log, the h2 code must be in the H2_SESSION_ST_BUSY state and 
the only cause I see is the same as you, namely an unexpected status from 
h2_session_read(), which should come via

status = ap_get_brigade(c->input_filters,
session->bbtmp, AP_MODE_READBYTES,
block? APR_BLOCK_READ : APR_NONBLOCK_READ,
H2MAX(APR_BUCKET_BUFF_SIZE, readlen));

block is 0 here and readlen should be (unless reconfigured via 
H2StreamMaxMemSize) 32kb.

Maybe you could just add a log line there to see what ap_get_brigade() returned 
there?

Strange.

-Stefan

> Am 13.10.2018 um 13:14 schrieb Rainer Jung :
> 
> Hi Stefan,
> 
> Am 10.10.2018 um 16:04 schrieb Stefan Eissing:
>>> Am 10.10.2018 um 15:06 schrieb Joe Orton :
>>> 
>>> I believe that t/modules/http2.t is dying in this:
>>> 
>>>my $old_ref = \&{ 'AnyEvent::TLS::_get_session' };
>>>*{ 'AnyEvent::TLS::_get_session' } = sub($$;$$) {
>>> 
>>> piece of magic which I don't understand but possibly needs updating for
>>> TLSv1.3? Session handling is different now... everything is broken.
>> I think there was no official way to add SNI to AnyEvent and I had to hack 
>> this. Unless anyone has another suggestion, I am in favour of removing the 
>> t/modules/http2.t again.
> 
> Note that if I manually send http2 requests using a recent curl, I get 
> failures as well (for 2.4.36).
> 
> The t/modules/http2.t indeed fails for each https test, even the simple first 
> one retrieving / and checking for status 200. One bug seems to me in the test 
> script, that it fails silently and simply notes that not all tests have run 
> at the end.
> 
> But if I only start the server using "t/TEST -start-httpd" and then run a 
> curl test request against the h2 port 8557, I get failures as well. The 
> server was build with TLS 1.3 support, but the failures occur with an 1.3 
> client but also with an 1.2 client (different builds of curl). I marked below 
> lines probably indicating the failure with  .
> 
> Here are details:
> 
> curl TLS 1.3 client output
> ==
> 
> *   Trying 127.0.0.1...
> * TCP_NODELAY set
> * Connected to localhost (127.0.0.1) port 8557 (#0)
> * ALPN, offering h2
> * ALPN, offering http/1.1
> * TLSv1.3 (OUT), TLS handshake, Client hello (1):
> * TLSv1.3 (IN), TLS handshake, Server hello (2):
> * TLSv1.3 (IN), TLS handshake, [no content] (0):
> * TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
> * TLSv1.3 (IN), TLS handshake, [no content] (0):
> * TLSv1.3 (IN), TLS handshake, Certificate (11):
> * TLSv1.3 (IN), TLS handshake, [no content] (0):
> * TLSv1.3 (IN), TLS handshake, CERT verify (15):
> * TLSv1.3 (IN), TLS handshake, [no content] (0):
> * TLSv1.3 (IN), TLS handshake, Finished (20):
> * TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
> * TLSv1.3 (OUT), TLS handshake, [no content] (0):
> * TLSv1.3 (OUT), TLS handshake, Finished (20):
> * SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
> * ALPN, server accepted to use h2
> * Server certificate:
> *  subject: C=US; ST=California; L=San Francisco; O=ASF; 
> OU=httpd-test/rsa-test; CN=localhost; emailAddress=test-...@httpd.apache.org
> *  start date: Oct 13 08:40:49 2018 GMT
> *  expire date: Oct 13 08:40:49 2019 GMT
> *  issuer: C=US; ST=California; L=San Francisco; O=ASF; OU=httpd-test; CN=ca; 
> emailAddress=test-...@httpd.apache.org
> *  SSL certificate verify result: self signed certificate in certificate 
> chain (19), continuing anyway.
> * Using HTTP2, server supports multi-use
> * Connection state changed (HTTP/2 confirmed)
> * Copying HTTP/2 data in stream buffer to connection buffer after upgrade: 
> len=0
> * TLSv1.3 (OUT), TLS app data, [no content] (0):
> * TLSv1.3 (OUT), TLS app data, [no content] (0):
> * TLSv1.3 (OUT), TLS app data, [no content] (0):
> * Using Stream ID: 1 (easy handle 0x26ab080)
> * TLSv1.3 (OUT), TLS app data, [no content] (0):
> > GET / HTTP/2
> > Host: localhost:8557
> > User-Agent: curl/7.61.1
> > Accept: */*
> >
> * TLSv1.3 (IN), TLS handshake, [no content] (0):
> * TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
> * TLSv1.3 (IN), TLS handshake, [no content] (0):
> * TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
> * TLSv1.3 (IN), TLS app data, [no content] (0):
> * Connection state changed (MAX_CONCURRENT_STREAMS == 100)!
> * TLSv1.3 (OUT), TLS app data, [no content] (0):
> * TLSv1.3 (IN), TLS app data, [no content] (0):
> * TLSv1.3 (IN), TLS alert, [no content] (0):
> * TLSv1.3 (IN), TLS alert, close notify (256):
> * Emp

Re: NOTICE: Intent to T 2.4.36

2018-10-11 Thread Stefan Eissing
Just pushed a fix to github.

> Am 11.10.2018 um 13:21 schrieb Stefan Eissing :
> 
> That is a change in trunk which has not been ported to our 2.4.x branch. 
> Since the github mod_http2 is intended for people who place  the module into 
> their 2.4.x server, I did not add a mpm version check for this - yet.
> 
> 
> 
>> Am 11.10.2018 um 13:17 schrieb Jim Jagielski :
>> 
>> BTW, and I'm sure you know this, that this fails w/ trunk:
>> 
>> % make
>> Making all in mod_http2
>> CC   mod_http2_la-h2_alt_svc.lo
>> CC   mod_http2_la-h2_bucket_beam.lo
>> CC   mod_http2_la-h2_bucket_eos.lo
>> CC   mod_http2_la-h2_config.lo
>> CC   mod_http2_la-h2_conn.lo
>> h2_conn.c:311:8: error: no member named 'data_in_input_filters' in 'struct 
>> conn_rec'; did you mean 'clogging_input_filters'?
>>   c->data_in_input_filters  = 0;
>>  ^
>>  clogging_input_filters
>> /usr/local2/apache2/include/httpd.h:1178:18: note: 'clogging_input_filters' 
>> declared here
>>   unsigned int clogging_input_filters:1;
>>^
>> h2_conn.c:312:8: error: no member named 'data_in_output_filters' in 'struct 
>> conn_rec'
>>   c->data_in_output_filters = 0;
>>   ~  ^
>> 2 errors generated.
> 



Re: NOTICE: Intent to T 2.4.36

2018-10-11 Thread Stefan Eissing
That is a change in trunk which has not been ported to our 2.4.x branch. Since 
the github mod_http2 is intended for people who place  the module into their 
2.4.x server, I did not add a mpm version check for this - yet.



> Am 11.10.2018 um 13:17 schrieb Jim Jagielski :
> 
> BTW, and I'm sure you know this, that this fails w/ trunk:
> 
> % make
> Making all in mod_http2
>  CC   mod_http2_la-h2_alt_svc.lo
>  CC   mod_http2_la-h2_bucket_beam.lo
>  CC   mod_http2_la-h2_bucket_eos.lo
>  CC   mod_http2_la-h2_config.lo
>  CC   mod_http2_la-h2_conn.lo
> h2_conn.c:311:8: error: no member named 'data_in_input_filters' in 'struct 
> conn_rec'; did you mean 'clogging_input_filters'?
>c->data_in_input_filters  = 0;
>   ^
>   clogging_input_filters
> /usr/local2/apache2/include/httpd.h:1178:18: note: 'clogging_input_filters' 
> declared here
>unsigned int clogging_input_filters:1;
> ^
> h2_conn.c:312:8: error: no member named 'data_in_output_filters' in 'struct 
> conn_rec'
>c->data_in_output_filters = 0;
>~  ^
> 2 errors generated.



Re: NOTICE: Intent to T 2.4.36

2018-10-11 Thread Stefan Eissing
Sry, forgot that in my mail. It's described in "build from git" in the 
README.md, my mistake.

> Am 11.10.2018 um 12:42 schrieb Jim Jagielski :
> 
> I suggest you add
> 
>   'autoreconf -i'
> 
> as a prelim step.



Re: [VOTE] Release httpd-2.4.36

2018-10-11 Thread Stefan Eissing
+1

 * macOS 10.14.0, XCode 10, OpenSSL 1.0.2: mod_http2, mod_md tests pass
 * macOS 10.14.0, XCode 10, OpenSSL 1.1.1: mod_http2, mod_md tests pass
 * ubuntu 18.04 LTS, gcc 7.3.0, openssl 1.1.0g: mod_http2 tests pass


> Am 10.10.2018 um 21:18 schrieb Daniel Ruggeri :
> 
> Hi, all;
>   Please find below the proposed release tarball and signatures:
> https://dist.apache.org/repos/dist/dev/httpd/
> 
> I would like to call a VOTE over the next few days to release this candidate 
> tarball as 2.4.36:
> [ ] +1: It's not just good, it's good enough!
> [ ] +0: Let's have a talk.
> [ ] -1: There's trouble in paradise. Here's what's wrong.
> 
> The computed digests of the tarball up for vote are:
> sha1: e40e7a879b84df860215b8a80f2a535534a1c4b4 *httpd-2.4.36.tar.gz
> sha256: ef788fb7c814acb2506a8b758a1a3f91f368f97bd4e6db16e98001f468e8e288 
> *httpd-2.4.36.tar.gz
> 
> -- 
> Daniel Ruggeri



Re: NOTICE: Intent to T 2.4.36

2018-10-11 Thread Stefan Eissing



> Am 10.10.2018 um 21:30 schrieb Jim Jagielski :
> 
> 
> 
>> On Oct 10, 2018, at 10:04 AM, Stefan Eissing  
>> wrote:
>> I have started to convert the existing h2 testsuite in 
>> https://svn.apache.org/repos/asf/httpd/test/mod_h2/trunk from shell scripts 
>> to pytest in the github repro. I have a pytest suite for mod_md also in its 
>> github. 
>> 
>> My hope is to, one day, make those part of a httpd test suite, probably just 
>> by combining the separate standalone ones. Then we could have 
>> 'modules/ABC/test' as optional part of a module with a defined way to 
>> trigger them.
>> 
> 
> That would be great because it seems that almost no one is using it, or has 
> been able to get that test suite to work. I know I haven't.

I would love to get feedback to mod-h2 tests from 
https://github.com/icing/mod_h2.

If you have an apxs and the apache/apr header files, pytest, nghttp2 and curl 
on your system, checkout and

> autoconf
> ./configure --with-apxs=
> make
> make test

On my machine, this gives:

> make test
/Applications/Xcode.app/Contents/Developer/usr/bin/make -C test/ test
rsync -a --exclude="*.in" e2e/conf/*.* 
/Users/sei/projects/mod-h2/test/gen/apache/conf
cd e2e && py.test
=== test 
session starts 

platform darwin -- Python 2.7.10, pytest-3.0.7, py-1.4.33, pluggy-0.4.0
rootdir: /Users/sei/projects/mod-h2/test/e2e, inifile:
collected 54 items

test_001_httpd_alive.py ..
test_002_curl_basics.py ..
test_003_curl_get.py .
test_004_curl_post.py 
test_100_conn_reuse.py .
test_101_ssl_reneg.py .
test_102_require.py ..
test_103_upgrade.py ...
test_200_header_invalid.py ...
test_201_header_conditional.py ..
test_202_trailer.py 
test_300_interim.py ...
test_400_push.py ..

 54 passed in 
13.25 seconds 

On my ubuntu image I get additionally:

= warnings 
summary =
test_001_httpd_alive.py::TestStore::()::test_001_02
  
/home/sei/.local/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:838:
 InsecureRequestWarning: Unverified HTTPS request is being made. Adding 
certificate verification is strongly advised. See: 
https://urllib3.readthedocs.io/en/latest/security.html
InsecureRequestWarning)

-- Docs: http://doc.pytest.org/en/latest/warnings.html

as I am not using 'mkcert' as of yet and certs are self-signed.

Re: NOTICE: Intent to T 2.4.36

2018-10-11 Thread Stefan Eissing
My view: 

Shipping the TLSv1.3 support has risks, but we also need to 
enable people to have the latest in transport security. 

We paid attention to keeping the behaviour with older versions
of openssl unchanged and our test cases confirm that this is the
case - as far or short as they go.

People linking their server (or shipping a distro) with openssl 1.1.1
or the relevant libressl version for TLSv1.3 opt in to share that
risk, consciously.

We can mitigate the risk of such changes with
a) better test suites
b) a broader test community

While no one is prevented from voting on a release candidate, the time
window is too short for some people, I believe. 

My experience with making mod_http2 and mod_md available standalone 
on github is:
- it's a platform for people who make "bleeding-edge" packages available
  to a range of people who like to run on the latest and greatest.
- it broadens the test base considerably
- it simplifies testing of fixes
- it simplifies analysis of bug reports against our 2.4.x releases as
  issues are often reported more than once and I can point people to
  the github release for checking their bug against fixes already made.

The httpd project is, right now, not offering this. People can check out
2.4.x, sure, but fixes often trickle only in just before a release. Also,
the 2.4.x branch is not intended for testing a "hopeful" fix. And testing
the dev branch is not the same either.

So, it seems to me, as we are missing a 2.4.x-candy branch that is CTR
and where from RTC merges  with votes are done to 2.4.x. From 2.4.x-candy
we could tag "release-candidates" and provide our usual tar balls in 
a pretty automated way. They'd have not CVE relevant changes and need no
big announcements. I imagine Daniels scripts can make them easily.

For 2.4.36 it would have meant that we'd had people testing TLSv1.3 for
weeks now, in much broader settings. And it would not have cost us much.

Cheers,

Stefan

> Am 10.10.2018 um 22:45 schrieb Gregg Smith :
> 
> FWIW, I've been running 2.4.36-dev at revision 1841586 for 19 days 35 minutes 
> as of this writing and I've seen no problems up to this point. Granted I only 
> get a few thousand hits a day and not millions but so far so good. Haven't 
> had many tls/1.3 but I would assume that's to be expected for another week or 
> two until Chrome 70 and Firefox 63 come out.
> 
> Now off to build .36
> 
> On 10/10/2018 1:29 PM, William A Rowe Jr wrote:
>> On Wed, Oct 10, 2018, 14:53 Mark Blackman  wrote:
>>> 
>>> Does the TLSv1.3 support need to be production ready?
>>> 
>>> TLSv1.3 is presumably an opt-in feature and as long as it doesn’t endanger
>>> existing behaviours, I would have assumed it’s relatively safe to release
>>> with caveats in the docs.
>>> Of course, once there’s more take-up of TLSv1.3, then the test suite needs
>>> to be useful. Getting real-world feedback about something completely new
>>> that doesn’t endanger existing behaviours outside of TLSv1.3 is probably
>>> worthwhile.
>>> 
>> Were it so easy...
>> It turns out httpd through 2.4.35 remain incompatible with changes to
>> openssl 1.1.1. This was disappointing from this project's perspective, the
>> issues are tracked on openssl project GitHub tickets.
>> If everything is good about this candidate, it should build and run against
>> 1.1.0, or 1.1.1, whether or not TLS 1.3 is enabled or avoided.
>> Ben Laurie last decade tried to address this with mod_tls, but mod_ssl
>> remains deeply tied to the internal behavior of libssl and libcrypto, to a
>> degree that it is effectively impossible to drop in 1.1.1 due to mechanical
>> changes in the protocol.
>> Dropping httpd 2.4.any into openssl 1.1.1 is a mess that several committers
>> have applied a great deal of attention to. We've undergone the same
>> problems with 1.1.0, 1.0.1, 1.0.0 and 0.9.8, so this didn't come as a
>> surprise.



Re: NOTICE: Intent to T 2.4.36

2018-10-10 Thread Stefan Eissing
Just in case of tl;dr:

The fix is proposed for backport, 1 vote is missing, would be nice to have in 
2.4.36

Thanks, Stefan

> Am 10.10.2018 um 10:47 schrieb Stefan Eissing :
> 
> I have a report of h2 missing an EOS flag on certain conditions related to 
> php served resources. Trying to reproduce. First such report came before 
> 2.4.35. I'd say, if I cannot progress on this today, then please go ahead and 
> tag. Will report later.
> 
> -Stefan
> 
>> Am 09.10.2018 um 22:29 schrieb Daniel Ruggeri :
>> 
>> Hi, all;
>>  I ran through my usual testing routine, this time with OpenSSL 1.1.1, but 
>> found several test failures. In the past, these issues have been isolated to 
>> my environment so I just wanted to drop a line to see if anyone has run the 
>> test suite against 2.4.x lately and can corroborate this result? If not, I 
>> can debug my environment.
>> 
>> Test Summary Report
>> ---
>> t/modules/http2.t (Wstat: 0 Tests: 24 Failed: 0)
>> Parse errors: Bad plan.  You planned 52 tests but ran 24.
>> t/security/CVE-2009-3555.t(Wstat: 0 Tests: 4 Failed: 2)
>> Failed tests:  3-4
>> t/ssl/basicauth.t (Wstat: 0 Tests: 4 Failed: 2)
>> Failed tests:  2-3
>> t/ssl/env.t   (Wstat: 0 Tests: 30 Failed: 15)
>> Failed tests:  16-30
>> t/ssl/extlookup.t (Wstat: 0 Tests: 4 Failed: 4)
>> Failed tests:  1-4
>> t/ssl/fakeauth.t  (Wstat: 0 Tests: 3 Failed: 2)
>> Failed tests:  2-3
>> t/ssl/ocsp.t  (Wstat: 0 Tests: 3 Failed: 1)
>> Failed test:  3
>> t/ssl/require.t   (Wstat: 0 Tests: 10 Failed: 3)
>> Failed tests:  2, 5, 9
>> t/ssl/varlookup.t (Wstat: 0 Tests: 83 Failed: 83)
>> Failed tests:  1-83
>> t/ssl/verify.t(Wstat: 0 Tests: 3 Failed: 1)
>> Failed test:  2
>> Files=186, Tests=8857, 101 wallclock secs ( 1.86 usr  0.28 sys + 48.46 cusr 
>> 11.08 csys = 61.68 CPU)
>> 
>> 
>> Versions at play were:
>> system:
>> kernel:
>>   name: Linux
>>   release: 3.16.0-4-amd64
>>   version: #1 SMP Debian 3.16.51-3 (2017-12-13)
>>   machine: x86_64
>> 
>> libraries:
>>   openssl: "1.1.1"
>>   openldap: "2.4.46"
>>   apr: "1.6.5"
>>   apr-util: "1.6.1"
>>   iconv: "1.2.2"
>>   brotli: "1.0.6"
>>   nghttp2: "1.34.0"
>>   zlib: "1.2.11"
>>   pcre: "8.42"
>>   libxml2: "2.9.8"
>>   php: "5.6.38"
>>   lua: "5.3.5"
>>   curl: "7.61.1"
>> 
>> 
>> Anything look obviously crazy/wrong?
>> 
>> -- 
>> Daniel Ruggeri
>> 
>> On 2018-10-09 06:36, Daniel Ruggeri wrote:
>>> Hi, all;
>>> Barring any major disagreement in the next several hours, I intend to
>>> T our next version later today or early tomorrow.
>>> Hooray for TLS 1.3!
>>> --
>>> Daniel Ruggeri
> 



Re: NOTICE: Intent to T 2.4.36

2018-10-10 Thread Stefan Eissing


> Am 10.10.2018 um 15:06 schrieb Joe Orton :
> 
> I believe that t/modules/http2.t is dying in this:
> 
>my $old_ref = \&{ 'AnyEvent::TLS::_get_session' };
>*{ 'AnyEvent::TLS::_get_session' } = sub($$;$$) {
> 
> piece of magic which I don't understand but possibly needs updating for 
> TLSv1.3? Session handling is different now... everything is broken.

I think there was no official way to add SNI to AnyEvent and I had to hack 
this. Unless anyone has another suggestion, I am in favour of removing the 
t/modules/http2.t again.

I have started to convert the existing h2 testsuite in 
https://svn.apache.org/repos/asf/httpd/test/mod_h2/trunk from shell scripts to 
pytest in the github repro. I have a pytest suite for mod_md also in its 
github. 

My hope is to, one day, make those part of a httpd test suite, probably just by 
combining the separate standalone ones. Then we could have 'modules/ABC/test' 
as optional part of a module with a defined way to trigger them.

Since each pytest module starts httpd and stops it again, the config files can 
be very local to the tests being done. That makes them quite easy to understand 
and startup times very short.

As for the certificates on a test host, I'd like to use 
https://github.com/FiloSottile/mkcert. That runs on MacOS, Linux and Windows. 
Not sure about the other OS we run on. More and more clients refuse to drop 
certificate verification or at least generate verbose warnings, so mkcert seems 
a good way to go.

-Stefan

Re: NOTICE: Intent to T 2.4.36

2018-10-10 Thread Stefan Eissing
Thanks, Joe. Tried to get it running on my Ubuntu 18.04 LTS image, but there I 
cannot even get the necessary Perl modules installed via CPAN.

I give up.

> Am 10.10.2018 um 15:06 schrieb Joe Orton :
> 
> On Wed, Oct 10, 2018 at 02:52:13PM +0200, Stefan Eissing wrote:
>> I cannot get the test framework to properly initialise any longer (MacOS 
>> 10.14):
>> 
>>> t/TEST -clean
>>> t/TEST
>> [warning] setting ulimit to allow core files
>> ulimit -c unlimited; /usr/bin/perl 
>> /Users/sei/projects/httpd/test/framework/trunk/t/TEST
>> [warning] generating SSL CA for asf
>> [   info] openssl req -new -x509 -keyout keys/ca.pem -out certs/ca.crt -days 
>> 365 -config conf/ca.cnf
>> Generating a 2048 bit RSA private key
>> ..+++
>> ..+++
>> writing new private key to 'keys/ca.pem'
>> -
>> problems making Certificate Request
>> 4620047980:error:0DFFF07A:asn1 encoding routines:CRYPTO_internal:first num 
>> too 
>> large:/BuildRoot/Library/Caches/com.apple.xbs/Sources/libressl/libressl-22.200.4/libressl-2.6/crypto/asn1/a_object.c:112:
>> 4620047980:error:0BFFF077:x509 certificate routines:CRYPTO_internal:invalid 
>> field 
>> name:/BuildRoot/Library/Caches/com.apple.xbs/Sources/libressl/libressl-22.200.4/libressl-2.6/crypto/x509/x509name.c:303:name=Email
>> [  error] configure() has failed:
>> system req -new -x509 -keyout keys/ca.pem -out certs/ca.crt -days 365 
>> -config conf/ca.cnf failed (exit status=1) at 
>> /Users/sei/projects/httpd/test/framework/trunk/Apache-Test/lib/Apache/TestSSLCA.pm
>>  line 216.
>> 
>> [warning] forcing Apache::TestConfig object save
>> [warning] run 't/TEST -clean' to clean up before continuing
>> 
>> Any tips?
> 
> Did you start from a fresh checkout?  I can't remember seeing that 
> particular error before but the whole thing is fragile as heck.
> 
> I believe that t/modules/http2.t is dying in this:
> 
>my $old_ref = \&{ 'AnyEvent::TLS::_get_session' };
>*{ 'AnyEvent::TLS::_get_session' } = sub($$;$$) {
> 
> piece of magic which I don't understand but possibly needs updating for 
> TLSv1.3? Session handling is different now... everything is broken.
> 
> The last output I get is:
> 
> ok 24
> test case: TC0001, expecting 200: GET https://localhost:8557/
> test case: VHOST000, expecting 200: GET https://localhost:8557/
> setting host_name to localhost:8557
> Failed 28/52 subtests 
> 
> so it looks like the perl script died completely somewhere around that 
> point.  My fedora 29 chroot has:
> 
> # rpm -q perl-AnyEvent openssl perl-interpreter
> perl-AnyEvent-7.14-7.fc29.x86_64
> openssl-1.1.1-3.fc29.x86_64
> perl-interpreter-5.28.0-423.fc29.x86_64
> 
> fwiw.



Re: NOTICE: Intent to T 2.4.36

2018-10-10 Thread Stefan Eissing
I cannot get the test framework to properly initialise any longer (MacOS 10.14):

> t/TEST -clean
> t/TEST
[warning] setting ulimit to allow core files
ulimit -c unlimited; /usr/bin/perl 
/Users/sei/projects/httpd/test/framework/trunk/t/TEST
[warning] generating SSL CA for asf
[   info] openssl req -new -x509 -keyout keys/ca.pem -out certs/ca.crt -days 
365 -config conf/ca.cnf
Generating a 2048 bit RSA private key
..+++
..+++
writing new private key to 'keys/ca.pem'
-
problems making Certificate Request
4620047980:error:0DFFF07A:asn1 encoding routines:CRYPTO_internal:first num too 
large:/BuildRoot/Library/Caches/com.apple.xbs/Sources/libressl/libressl-22.200.4/libressl-2.6/crypto/asn1/a_object.c:112:
4620047980:error:0BFFF077:x509 certificate routines:CRYPTO_internal:invalid 
field 
name:/BuildRoot/Library/Caches/com.apple.xbs/Sources/libressl/libressl-22.200.4/libressl-2.6/crypto/x509/x509name.c:303:name=Email
[  error] configure() has failed:
system req -new -x509 -keyout keys/ca.pem -out certs/ca.crt -days 365 -config 
conf/ca.cnf failed (exit status=1) at 
/Users/sei/projects/httpd/test/framework/trunk/Apache-Test/lib/Apache/TestSSLCA.pm
 line 216.

[warning] forcing Apache::TestConfig object save
[warning] run 't/TEST -clean' to clean up before continuing

Any tips?

> Am 10.10.2018 um 14:30 schrieb Joe Orton :
> 
> On Tue, Oct 09, 2018 at 03:29:49PM -0500, Daniel Ruggeri wrote:
>> Hi, all;
>>   I ran through my usual testing routine, this time with OpenSSL 1.1.1, but
>> found several test failures. In the past, these issues have been isolated to
>> my environment so I just wanted to drop a line to see if anyone has run the
>> test suite against 2.4.x lately and can corroborate this result? If not, I
>> can debug my environment.
> 
> TLSv1.3 testing is still a mess with OpenSSL 1.1.1, sorry.  I have 
> updated the test suite just now to disable TLSv1.3 testing for most 
> people.  We need updates to Net::SSLeay (the latest upstream has the 
> patch) and IO::Socket::SSL, but the latter is not patched upstream, so I 
> can't make an accurate test for that yet.
> 
> At worst, forcibly testing with:
> 
>  ./t/TEST -sslproto 'all -TLSv1.2'
> 
> should now be possible.
> 
> (If using an existing check-out of the test suite don't forget to re-run 
> "make" before running ./t/TEST -conf to regenerate the config...)
> 
> Let me know if that's not made any difference for you.
> 
> I don't know why t/modules/http2.t is failing but I see that here too.
> 
> Regards, Joe
> 
> 
>> 
>> Test Summary Report
>> ---
>> t/modules/http2.t (Wstat: 0 Tests: 24 Failed: 0)
>>  Parse errors: Bad plan.  You planned 52 tests but ran 24.
>> t/security/CVE-2009-3555.t(Wstat: 0 Tests: 4 Failed: 2)
>>  Failed tests:  3-4
>> t/ssl/basicauth.t (Wstat: 0 Tests: 4 Failed: 2)
>>  Failed tests:  2-3
>> t/ssl/env.t   (Wstat: 0 Tests: 30 Failed: 15)
>>  Failed tests:  16-30
>> t/ssl/extlookup.t (Wstat: 0 Tests: 4 Failed: 4)
>>  Failed tests:  1-4
>> t/ssl/fakeauth.t  (Wstat: 0 Tests: 3 Failed: 2)
>>  Failed tests:  2-3
>> t/ssl/ocsp.t  (Wstat: 0 Tests: 3 Failed: 1)
>>  Failed test:  3
>> t/ssl/require.t   (Wstat: 0 Tests: 10 Failed: 3)
>>  Failed tests:  2, 5, 9
>> t/ssl/varlookup.t (Wstat: 0 Tests: 83 Failed: 83)
>>  Failed tests:  1-83
>> t/ssl/verify.t(Wstat: 0 Tests: 3 Failed: 1)
>>  Failed test:  2
>> Files=186, Tests=8857, 101 wallclock secs ( 1.86 usr  0.28 sys + 48.46 cusr
>> 11.08 csys = 61.68 CPU)
>> 
>> 
>> Versions at play were:
>> system:
>>  kernel:
>>name: Linux
>>release: 3.16.0-4-amd64
>>version: #1 SMP Debian 3.16.51-3 (2017-12-13)
>>machine: x86_64
>> 
>>  libraries:
>>openssl: "1.1.1"
>>openldap: "2.4.46"
>>apr: "1.6.5"
>>apr-util: "1.6.1"
>>iconv: "1.2.2"
>>brotli: "1.0.6"
>>nghttp2: "1.34.0"
>>zlib: "1.2.11"
>>pcre: "8.42"
>>libxml2: "2.9.8"
>>php: "5.6.38"
>>lua: "5.3.5"
>>curl: "7.61.1"
>> 
>> 
>> Anything look obviously crazy/wrong?
>> 
>> -- 
>> Daniel Ruggeri
>> 
>> On 2018-10-09 06:36, Daniel Ruggeri wrote:
>>> Hi, all;
>>> Barring any major disagreement in the next several hours, I intend to
>>> T our next version later today or early tomorrow.
>>> 
>>> Hooray for TLS 1.3!
>>> --
>>> Daniel Ruggeri



Re: svn commit: r1843434 - /httpd/httpd/branches/2.4.x/STATUS

2018-10-10 Thread Stefan Eissing
If I could win 2 people to review and vote for this small patch? The reporter 
has confirmed that it resolves his issue 
(https://github.com/icing/mod_h2/issues/170). Thanks!

Cheers,

Stefan

> Am 10.10.2018 um 14:13 schrieb ic...@apache.org:
> 
> Author: icing
> Date: Wed Oct 10 12:13:41 2018
> New Revision: 1843434
> 
> URL: http://svn.apache.org/viewvc?rev=1843434=rev
> Log:
> adding proposal to backport h2 eos handling fix
> 
> Modified:
>httpd/httpd/branches/2.4.x/STATUS
> 
> Modified: httpd/httpd/branches/2.4.x/STATUS
> URL: 
> http://svn.apache.org/viewvc/httpd/httpd/branches/2.4.x/STATUS?rev=1843434=1843433=1843434=diff
> ==
> --- httpd/httpd/branches/2.4.x/STATUS (original)
> +++ httpd/httpd/branches/2.4.x/STATUS Wed Oct 10 12:13:41 2018
> @@ -195,6 +195,11 @@ PATCHES PROPOSED TO BACKPORT FROM TRUNK:
>  2.4.x patch: svn merge -c 1839303,1843290 ^/httpd/httpd/trunk . 
>  +1: jailletc36 (by inspection), ylavic
> 
> +  *) mod_http2: fixing an issue that h2 stream do not properly EOS when
> + the bucket is missing in the handler response.
> + trunk patch: http://svn.apache.org/r1843426
> + 2.4.x patch: 
> https://svn.apache.org/repos/asf/httpd/httpd/patches/2.4.x/h2-eos-fix.patch
> + +1: icing, 
> 
> PATCHES/ISSUES THAT ARE BEING WORKED
>   [ New entries should be added at the START of the list ]
> 
> 



Re: NOTICE: Intent to T 2.4.36

2018-10-10 Thread Stefan Eissing
I have a report of h2 missing an EOS flag on certain conditions related to php 
served resources. Trying to reproduce. First such report came before 2.4.35. 
I'd say, if I cannot progress on this today, then please go ahead and tag. Will 
report later.

-Stefan

> Am 09.10.2018 um 22:29 schrieb Daniel Ruggeri :
> 
> Hi, all;
>   I ran through my usual testing routine, this time with OpenSSL 1.1.1, but 
> found several test failures. In the past, these issues have been isolated to 
> my environment so I just wanted to drop a line to see if anyone has run the 
> test suite against 2.4.x lately and can corroborate this result? If not, I 
> can debug my environment.
> 
> Test Summary Report
> ---
> t/modules/http2.t (Wstat: 0 Tests: 24 Failed: 0)
>  Parse errors: Bad plan.  You planned 52 tests but ran 24.
> t/security/CVE-2009-3555.t(Wstat: 0 Tests: 4 Failed: 2)
>  Failed tests:  3-4
> t/ssl/basicauth.t (Wstat: 0 Tests: 4 Failed: 2)
>  Failed tests:  2-3
> t/ssl/env.t   (Wstat: 0 Tests: 30 Failed: 15)
>  Failed tests:  16-30
> t/ssl/extlookup.t (Wstat: 0 Tests: 4 Failed: 4)
>  Failed tests:  1-4
> t/ssl/fakeauth.t  (Wstat: 0 Tests: 3 Failed: 2)
>  Failed tests:  2-3
> t/ssl/ocsp.t  (Wstat: 0 Tests: 3 Failed: 1)
>  Failed test:  3
> t/ssl/require.t   (Wstat: 0 Tests: 10 Failed: 3)
>  Failed tests:  2, 5, 9
> t/ssl/varlookup.t (Wstat: 0 Tests: 83 Failed: 83)
>  Failed tests:  1-83
> t/ssl/verify.t(Wstat: 0 Tests: 3 Failed: 1)
>  Failed test:  2
> Files=186, Tests=8857, 101 wallclock secs ( 1.86 usr  0.28 sys + 48.46 cusr 
> 11.08 csys = 61.68 CPU)
> 
> 
> Versions at play were:
> system:
>  kernel:
>name: Linux
>release: 3.16.0-4-amd64
>version: #1 SMP Debian 3.16.51-3 (2017-12-13)
>machine: x86_64
> 
>  libraries:
>openssl: "1.1.1"
>openldap: "2.4.46"
>apr: "1.6.5"
>apr-util: "1.6.1"
>iconv: "1.2.2"
>brotli: "1.0.6"
>nghttp2: "1.34.0"
>zlib: "1.2.11"
>pcre: "8.42"
>libxml2: "2.9.8"
>php: "5.6.38"
>lua: "5.3.5"
>curl: "7.61.1"
> 
> 
> Anything look obviously crazy/wrong?
> 
> -- 
> Daniel Ruggeri
> 
> On 2018-10-09 06:36, Daniel Ruggeri wrote:
>> Hi, all;
>> Barring any major disagreement in the next several hours, I intend to
>> T our next version later today or early tomorrow.
>> Hooray for TLS 1.3!
>> --
>> Daniel Ruggeri



Re: Wherefor 2.4.36?

2018-10-08 Thread Stefan Eissing



> Am 07.10.2018 um 03:16 schrieb Daniel Ruggeri :
> 
> Actually, I'm glad you asked. I committed after 2.4.35 to T 2.4.36 soon 
> after. I'm happy to do that ASAP if there are no objections.
> 
> What say you, fellow devs? How about next week?
> -- 
> Daniel Ruggeri
> 
> On October 6, 2018 7:53:58 PM CDT, Michael-Fever  wrote:
> 
> Aww, all I care about is getting 2.4.36 going so I can say I have TLS 1.3
> supported with my h2.  LOL, no but seriously, is 2.4.36 stable enough to be
> using?

+1

Very happy to see that. Thanks, Daniel!


Re: Minimum OpenSSL requirements for mod_md

2018-09-24 Thread Stefan Eissing



> Am 24.09.2018 um 14:58 schrieb Astrid Malo :
> 
> On Mon, 24 Sep 2018 14:42:06 +0200
> Rainer Jung  wrote:
> 
>> Should we document that requirement somehow, because our non-mod_md 
>> OpenSSL requirement is still at 0.9.8a. IMHO there's no need to "fix" 
>> the higher requirement in mod_md, because it is pretty fresh and 
>> probably there's no need to support it with ancient OpenSSL.
> 
> Yes, of course. Requirements differing from the general ones have to be
> added to the documentation. This should be no big deal. Just make it
> prominent somewhere where people read if before the installation :-)
> 
> kess

+1



Re: svn commit: r1841573 - in /httpd/httpd/branches/2.4.x: ./ CHANGES STATUS docs/manual/mod/mod_ssl.xml modules/ssl/mod_ssl.c modules/ssl/ssl_engine_config.c modules/ssl/ssl_engine_init.c modules/ssl

2018-09-21 Thread Stefan Eissing
822624,1822849,1822858,1822878-1822879,1822883,1822931,1823047,1823179,1823412,1823415-1823416,1823482,1823564,1823572,1823575,1823886,1824176,1824303,1824332,1824336,1824343,1824381,1824390,1824454,1824460,1824463-1824464,1824482,1824497,1824811,1824862,1824877,1824973,1825147,1825169,1825368,1825370,1825467,1825504,1826207,1826556,1826686-1826687,1826845,1826847,1826973,1826995,1827001,1827166,1827196,1827362,1827366,1827374,1827599,1827604,1827654,1827671,1827783,1827865,1827912,1827924,1827992,1828210,1828222,1828232,1828390,1828485,1828493,1828669,1828687,182872
>> 0,1828723,1828790-1828792,1828879,1828890,1828912,1828920,1828926-1828927,1829038-1829039,1829513,1829557,1829573,1829645,1829657,1830523,1830562,1830744,1830746,1830943-1830944,1831231,1831591,1831772,1831800,1832198,1832200,1832277,1832280,1832317,1832351,1832500,1832580-1832581,1832934,1832937,1832951,1832991,1833014,1833588-1833589,1833827,1833875-1833876,1834012-1834013,1834209,1834226,1834318,1834470,1835094,1835118,1835287,1836095,1836154,1836276,1836287,1836381-1836383,1836386,1836469,1836603,1837130,1837357,1837588-1837590,1837595,1838937,1839780,1839920,1839946,1840010,1840582,1840585,1840710,1840776,1841218
>> 
>> Modified: httpd/httpd/branches/2.4.x/CHANGES
>> URL: 
>> http://svn.apache.org/viewvc/httpd/httpd/branches/2.4.x/CHANGES?rev=1841573=1841572=1841573=diff
>> ==
>> --- httpd/httpd/branches/2.4.x/CHANGES [utf-8] (original)
>> +++ httpd/httpd/branches/2.4.x/CHANGES [utf-8] Fri Sep 21 12:14:05 2018
>> @@ -1,6 +1,19 @@
>> -*- coding: utf-8 -*-
>> Changes with Apache 2.4.36
>> 
>> +  *) mod_ssl: add experimental support for TLSv1.3 (tested with OpenSSL 
>> v1.1.1-pre9. 
>> + SSL(Proxy)CipherSuite now has an optional first parameter for the 
>> protocol the ciphers are for.
>> + Directive "SSLVerifyClient" now triggers certificate retrieval from 
>> the client.
>> + Verifying the client fails exactly the same for HTTP/2 connections for 
>> all SSL protocols,
>> + as this would need to trigger the master connection thread - which we 
>> do not support
>> + right now.
>> + Renegotiation of ciphers is intentionally ignored for TLSv1.3 
>> connections. "SSLCipherSuite"
>> + does not allow to specify TLSv1.3 ciphers in a directory context 
>> (because it cannot work) and
>> + TLSv1.2 or lower ciphers are not relevant for 1.3, as cipher suites 
>> are completely separate.
>> + Sites which make use of such TLSv1.2 feature need to evaluate 
>> carefully if or how they 
>> + can match their needs onto the TLSv1.3 protocol.
>> + [Yann Ylavic, Stefan Eissing]
>> +
>>  *) mod_auth_basic: Be less tolerant when parsing the credencial. Only spaces
>> should be accepted after the authorization scheme. \t are also tolerated.
>> [Christophe Jaillet]
>> 
>> Modified: httpd/httpd/branches/2.4.x/STATUS
>> URL: 
>> http://svn.apache.org/viewvc/httpd/httpd/branches/2.4.x/STATUS?rev=1841573=1841572=1841573=diff
>> ==
>> --- httpd/httpd/branches/2.4.x/STATUS (original)
>> +++ httpd/httpd/branches/2.4.x/STATUS Fri Sep 21 12:14:05 2018
>> @@ -124,26 +124,6 @@ RELEASE SHOWSTOPPERS:
>> PATCHES ACCEPTED TO BACKPORT FROM TRUNK:
>>  [ start all new proposals below, under PATCHES PROPOSED. ]
>> 
>> -   *) Add TLSv1.3 support to mod_ssl:
>> -  trunk: http://svn.apache.org/r1839946
>> - http://svn.apache.org/r1839920
>> - http://svn.apache.org/r1833589
>> - http://svn.apache.org/r1833588
>> - http://svn.apache.org/r1828723
>> - http://svn.apache.org/r1828720
>> - http://svn.apache.org/r1828222
>> - http://svn.apache.org/r1827992
>> - http://svn.apache.org/r1827924
>> - http://svn.apache.org/r1827912
>> - http://svn.apache.org/r1828790
>> - http://svn.apache.org/r1828791
>> - http://svn.apache.org/r1828792
>> - http://svn.apache.org/r1840585
>> - http://svn.apache.org/r1840710
>> - http://svn.apache.org/r1841218
>> -  2.4.x branch: svn merge ^/httpd/httpd/branches/tlsv1.3-for-2.4.x
>> -  +1: icing, jorton, minfrin (tested on openssl-1.0.2j and 
>> openssl-1.1.1)
>> -
>> 
>> 
>> PATCHES PROPOSED TO BACKPORT FROM TRUNK:
>> 
>> Modified: httpd/httpd/branches/2.4.x/docs/manual/mod/mod_ssl.xml
>> U

  1   2   3   4   5   6   7   8   9   10   >