Re: async mod_proxy_http

[Eric Covener]

On Wed, Sep 12, 2018 at 7:59 PM Yann Ylavic  wrote:
>
> On Wed, Sep 12, 2018 at 5:53 PM Eric Covener  wrote:
> >
> > Forking from the Cool Stuff thread.
> >
> > Have you noticed that the wstunnell stuff makes the suspended count in
> > the MPM grow? There is no API for us to tell the MPM that when we get
> > the socket-activity callback that we are "resuming" something.
> >
> > (going from vague recollection)
>
> It seems that we increment it once when the handler returns SUSPENDED,
> and decrement it once per connection too in proxy_wstunnel_finish().

This didn't even ring a bell :)

But won't we return SUSPENDED many times to a single finish?

>
> However, looks like there is unnecessary churn
> proxy_wstunnel_finish(), including a double close since the MPM will
> also finally close the client connection. How about something like the
> attached?

Looks reasonable.

Re: async mod_proxy_http

[Yann Ylavic]

On Wed, Sep 12, 2018 at 5:53 PM Eric Covener  wrote:
>
> Forking from the Cool Stuff thread.
>
> Have you noticed that the wstunnell stuff makes the suspended count in
> the MPM grow? There is no API for us to tell the MPM that when we get
> the socket-activity callback that we are "resuming" something.
>
> (going from vague recollection)

It seems that we increment it once when the handler returns SUSPENDED,
and decrement it once per connection too in proxy_wstunnel_finish().

However, looks like there is unnecessary churn
proxy_wstunnel_finish(), including a double close since the MPM will
also finally close the client connection. How about something like the
attached?

Regards,
Yann.
Index: modules/proxy/mod_proxy_wstunnel.c
===
--- modules/proxy/mod_proxy_wstunnel.c	(revision 1840709)
+++ modules/proxy/mod_proxy_wstunnel.c	(working copy)
@@ -155,15 +155,14 @@ static int proxy_wstunnel_pump(ws_baton_t *baton,
 
 static void proxy_wstunnel_finish(ws_baton_t *baton)
 { 
+conn_rec *c = baton->r->connection;
 ap_log_rerror(APLOG_MARK, APLOG_TRACE1, 0, baton->r, "proxy_wstunnel_finish");
 baton->proxy_connrec->close = 1; /* new handshake expected on each back-conn */
-baton->r->connection->keepalive = AP_CONN_CLOSE;
 ap_proxy_release_connection(baton->scheme, baton->proxy_connrec, baton->r->server);
-ap_finalize_request_protocol(baton->r);
-ap_lingering_close(baton->r->connection);
-apr_socket_close(baton->client_soc);
-ap_mpm_resume_suspended(baton->r->connection);
+ap_finalize_request_protocol(baton->r); /* send EOS */
+c->aborted = 1; /* nothing more on the client connection */
 ap_process_request_after_handler(baton->r); /* don't touch baton or r after here */
+ap_mpm_resume_suspended(c);
 }
 
 /* If neither socket becomes readable in the specified timeout,
Index: server/mpm/event/event.c
===
--- server/mpm/event/event.c	(revision 1840709)
+++ server/mpm/event/event.c	(working copy)
@@ -1273,6 +1273,12 @@ static apr_status_t event_resume_suspended (conn_r
 apr_atomic_dec32(_count);
 c->suspended_baton = NULL;
 
+if (c->aborted) {
+apr_socket_close(ap_get_conn_socket(c));
+ap_queue_info_push_pool(worker_queue_info, cs->p);
+return OK;
+}
+
 cs->queue_timestamp = apr_time_now();
 cs->pfd.reqevents = (
 cs->pub.sense == CONN_SENSE_WANT_READ ? APR_POLLIN :

Re: 2.4.35 in Sept?

[Jim Jagielski]

> On Sep 12, 2018, at 4:14 PM, William A Rowe Jr  wrote:
> 
> Since we still have no schema to solve the project maintenance side of
> shipping source code

Sorry, but we have this thing called creating test tarballs and calling for a 
vote. I am sure you remember that.

The idea is that people actually take the time to download the tarballs, build 
a version of httpd for their use and then perform testing on said version such 
that they can vote on whether to release it or not. That testing entails such 
activities as running it through our test framework, putting it in a dev/prod 
environment to see how it works, etc... That is a schema that we have had 
for... let me see... years and decades.

What improvements do you have to suggest to improve upon this? Do you recommend 
a longer vote time? Do you recommend beta and/or release-candidates? Do you 
recommend that the 1st born of all voters be held in a camp until the release 
has "proven" itself to be up to your satisfaction ensuring that said voters 
have "skin in the game"?

It is better to light a single candle than to curse the darkness.

Re: 2.4.35 in Sept?

[Jim Jagielski]

> On Sep 12, 2018, at 4:14 PM, William A Rowe Jr  wrote:
> 
> On Tue, Sep 11, 2018 at 8:01 AM Jim Jagielski  > wrote:
> If there is still interest, there are a handful of useful backports in STATUS 
> that could use review, testing and voting :-)
> 
> Unless the goal is to replace one set of regressions with a new set 
> of regressions, it's past due to tag 2.4.35 before "useful" returns as 
> the de facto criteria again on 2.4.x.
> 

Certainly that is up to the RM and the people reviewing, testing and voting on 
the backports...

Re: 2.4.35 in Sept?

[William A Rowe Jr]

On Tue, Sep 11, 2018 at 8:01 AM Jim Jagielski  wrote:

> If there is still interest, there are a handful of useful backports in
> STATUS that could use review, testing and voting :-)
>

Unless the goal is to replace one set of regressions with a new set
of regressions, it's past due to tag 2.4.35 before "useful" returns as
the de facto criteria again on 2.4.x.

Because the TLS 1.3 patchset is significantly bigger, I'd appreciate
an httpd release prior to merging that effort. So we have 2.4.35 entirely
usable, and soon thereafter, 2.4.36 with TLS 1.3, and whatever "useful"
additions people want to pile on (as observed of the 2.4/trunk diff).

Since we still have no schema to solve the project maintenance side of
shipping source code, getting 2.4.35 with principally regression-fixes out
the door before new regressions are added seems wise. I'm happy to
offer to RM this coming Monday if nobody beats me to it, if we believe
all of the known regressions have been closed.

Is anyone aware of any lingering regression fixes to apply for 2.4.35
in the immediate-term?

September for 2.4.36 w/TLS 1.3 still seems plausible. If we really want
to prove up that patch set, 2.5.1-alpha sooner rather than later might
make good sense, and would likely attract more attention than a typical
alpha owing to draft TLS 1.3 support alone.

Re: TLSv1.3 supprt for 2.4.x?

[Yann Ylavic]

On Wed, Sep 12, 2018 at 3:17 PM Joe Orton  wrote:
>
> On Tue, Sep 11, 2018 at 03:39:42PM +0200, Yann Ylavic wrote:
> > On Tue, Sep 11, 2018 at 12:13 PM Joe Orton  wrote:
> > >
> > > Does anybody have successful test results with post-handshake auth?  I'm
> > > testing against Fedora's OpenSSL 1.1.1pre9 which has merged the changes
> > > for https://github.com/openssl/openssl/issues/6933
> >
> > Just tried trunk+openssl-1.1.1pre9 (SSLProtocol TLSv1.3 and
> > SSLVerifyClient require), with both firefox and s_client (w/ and w/o
> > -enable_pha) and can't reproduce the hang.
> >
> > What's your client tooling?
>
> Have you tried the test suite with trunk+pre9?  I haven't had time to
> test this directly, but with Fedora 29's pre9 + tls1.3-2.4.x branch I am
> down to:
>
> $ ./t/TEST t/ssl/
> ...
> Failed 5/14 test programs. 12/331 subtests failed.

With patch [1] (SSL_CTX_clear_mode for AUTO_RETRY) and your latest
framework changes, the only errors remaining are:

t/modules/http2.t ... Failed 28/52 subtests
("Parse errors: Bad plan.  You planned 52 tests but ran 24", tests
skipped because of TLS-1.3??).

t/ssl/proxy.t .. 3/? # Failed test 3 in t/ssl/proxy.t at line 63
# Failed test 5 in t/ssl/proxy.t at line 75
# Failed test 6 in t/ssl/proxy.t at line 89
# Failed test 7 in t/ssl/proxy.t at line 95
t/ssl/proxy.t .. 113/? # Failed test 116 in t/ssl/proxy.t at line 63 fail #2
# Failed test 118 in t/ssl/proxy.t at line 75 fail #2
# Failed test 119 in t/ssl/proxy.t at line 89 fail #2
# Failed test 120 in t/ssl/proxy.t at line 95 fail #2
t/ssl/proxy.t .. Failed 8/172 subtests
(didn't look at the details)

For the ssl/proxy ones, I tried with patch [2]
(SSL_CTX_set_post_handshake_auth):
t/ssl/proxy.t .. 1/? # Failed test 4 in t/ssl/proxy.t at line 69
t/ssl/proxy.t .. 108/? # Failed test 117 in t/ssl/proxy.t at line 69 fail #2
t/ssl/proxy.t .. Failed 2/172 subtests
(better but not really that yet. Anyway I don't really understand why
we'd do that, and how it helps Perl, given that it's supposed to be
opt-in for compatibility, and AFAICT ssl/proxy tests don't use
TLS-1.3...)

>
> 1. disable AUTO_RETRY to fix PHA
> https://github.com/openssl/openssl/issues/7178#issuecomment-420300988
>
> 2. too much code has been moved out of ssl_hook_Access(), the
> FakeBasicAuth code & requires checking needs to come back
>
> 3. perl client side updates needed in IO::Socket:SSL and Net::SSLeay to
> re-enable PHA because of https://github.com/openssl/openssl/issues/6933
>
> I will commit (1) and (2) today/tomorrow. I think (3) is, um, bizarre,
> but I'm not sure whether it is simpler to fight OpenSSL API design
> decisions or change every SSL client in the world to cope with them,
> probably the latter.

I can't grok that change needed on the client-side either :/

Regards,
Yann.

[1]:
Index: modules/ssl/ssl_engine_init.c
===
--- modules/ssl/ssl_engine_init.c(revision 1840709)
+++ modules/ssl/ssl_engine_init.c(working copy)
@@ -786,6 +786,10 @@ static apr_status_t ssl_init_ctx_protocol(server_r
 SSL_CTX_set_mode(ctx, SSL_MODE_RELEASE_BUFFERS);
 #endif

+#ifdef SSL_MODE_AUTO_RETRY
+SSL_CTX_clear_mode(ctx, SSL_MODE_AUTO_RETRY);
+#endif
+
 return APR_SUCCESS;
 }
--

[2]:
Index: modules/ssl/ssl_engine_init.c
===
--- modules/ssl/ssl_engine_init.c(revision 1840709)
+++ modules/ssl/ssl_engine_init.c(working copy)
@@ -1553,6 +1557,11 @@ static apr_status_t ssl_init_proxy_certs(server_re

 SSL_CTX_set_client_cert_cb(mctx->ssl_ctx,
ssl_callback_proxy_cert);
+#if OPENSSL_VERSION_NUMBER >= 0x10101000L
+if (/*mctx->enable_pha*/1) {
+SSL_CTX_set_post_handshake_auth(mctx->ssl_ctx, 1);
+}
+#endif

 if (!(pkp->cert_file || pkp->cert_path)) {
 return APR_SUCCESS;
--

Re: svn commit: r1840585 - in /httpd/httpd/trunk: docs/log-message-tags/next-number modules/ssl/ssl_engine_kernel.c

[Joe Orton]

On Wed, Sep 12, 2018 at 03:11:48PM +0200, Stefan Eissing wrote:
> How much have your testings now proceeded? Yann reported interop with
> firefox for him against trunk. Did you manage to track down your
> problems? Something missing in the branch?

Right now for me there is only the ssl_hook_Access() fix to commit, and 
with that in I am seeing just one failure in all of t/ssl in the test 
suite:

Test Summary Report
---
t/ssl/ocsp.t (Wstat: 0 Tests: 3 Failed: 1)
  Failed test:  3
Files=14, Tests=331, 10 wallclock secs ( 0.18 usr  0.03 sys +  4.38 cusr  
1.15 csys =  5.74 CPU)
Result: FAIL
Failed 1/14 test programs. 1/331 subtests failed.

The remaining failure is the test case trying to detect the "certificate 
revoked" TLS alert, which is not getting exposed through the Perl 
TLS/HTTP error handling in the same way as it was before.  The server is 
sending the alert so it looks like a false positive.

Regards, Joe

async mod_proxy_http

[Eric Covener]

Forking from the Cool Stuff thread.

Have you noticed that the wstunnell stuff makes the suspended count in
the MPM grow? There is no API for us to tell the MPM that when we get
the socket-activity callback that we are "resuming" something.

(going from vague recollection)

-- 
Eric Covener
cove...@gmail.com

Re: Cool Stuff In trunk: (Was: Re: 2.4.x and 2.6.x ... next steps)

[Yann Ylavic]

On Wed, Sep 12, 2018 at 2:57 PM Stefan Eissing
 wrote:
>
>
> > Am 12.09.2018 um 14:48 schrieb Jim Jagielski :
> >
> > As I said before, the main assumption in my suggestion is that there are 
> > things in trunk that "really should" be in some releasable version, stuff 
> > significant enough to warrant the work, but is "impossible" to be 
> > backported to 2.4.
> >
> > If there are no real significant-but-impossible-to-backport features in 
> > trunk, then the proposal itself is moot.
> >
> > So let's think about it: What is currently in trunk that is a pretty 
> > significant improvement? Then ask if it is directly backportable. Certainly 
> > the effort in backporting from trunk to 2.4.x is much less than the effort 
> > in spinning out 2.6.x and considering all things, that should be the 
> > primary flow.
>
> There are things I'd like to do for 2.5.x-to-become-2.6 releases that I 
> cannot to in 2.4.x and will not do before that. I assume this holds also true 
> for others.

+1

FWIW I've got WIP on async mod_proxy_http (based on wstunnel/event "go
async" mode, with common functions put in proxy_util, the pump/forward
loop from PR 61616, websocket proxying friendly).
Of course I'll propose it on dev@ first, and if it's received
positively that could make great stuff for 2.6, but not for 2.4 where
modules are not really prepared to be changed their running thread
underneath them...

>
> To put it another way: current trunk is dead code to me. Only a stopover for 
> 2.4.x (aka release version). For the last three years, it was just in the way.

That can't/shouldn't be, trunk is where we do improvements AND the
next released version.
Once we take less care of 2.4 compat, things become much easier IMHO.

>
> Or another way: I am too old to commit to trunk only. ;)

I'm sure that trunk + all of your (and others) pending stuff made to
2.6 and you go for a new youth :)

Re: 2.4.x and 2.6.x ... next steps

[Jim Jagielski]

> On Sep 12, 2018, at 10:27 AM, Stefan Eissing  
> wrote:
> 
> I feel myself in agreement with Bill that trunk needs to be where 2.5.x is 
> born.
> 

It is. That should be clear in the proposal.

What should also be clear is that there is a LOT in trunk that should be in 
2.4.x and has nothing to do with 2.5/2.6/3.0/next-gen. So backporting that 
stuff HELPS midwifing 2.5/2.6!

Re: Cool Stuff In trunk: (Was: Re: 2.4.x and 2.6.x ... next steps)

[Jim Jagielski]

Thanks, this is useful.

At first blush, this looks like there is a crap-ton of stuff in trunk than can, 
and should, be quickly and easily backported to 2.4 asap!!

> On Sep 12, 2018, at 10:06 AM, William A Rowe Jr  wrote:
> 
> On Wed, Sep 12, 2018 at 7:49 AM Jim Jagielski  > wrote:
> As I said before, the main assumption in my suggestion is that there are 
> things in trunk that "really should" be in some releasable version
> 
> Everything in trunk is now digested into three groups of commits, for 
> inspection. These don't include whitespace changes, so the resulting files 
> may be mis-indented and otherwise skewed, but we are looking for changes, not 
> insignificant formatting;
> 
> Just the docs (sources, and generated)
> http://svn.apache.org/viewvc?view=revision=1840682 
> 
> http://svn.apache.org/viewvc?view=revision=1840684 
> 
> 
> Just win32 generated build gunk (presuming vcproj and CMake are both winners, 
> there is a discussion to be had on .mak files before this gap is addressed)
> http://svn.apache.org/viewvc?view=revision=1840687 
> 
> 
> Everything else (all the interesting stuff)
> http://svn.apache.org/viewvc?view=revision=1840688 
>  
> 
> I expect committers will be mostly interested in the last link above.
> 

Re: 2.4.x and 2.6.x ... next steps

[William A Rowe Jr]

On Wed, Sep 12, 2018, 09:19 Graham Leggett  wrote:

> On 12 Sep 2018, at 15:39, William A Rowe Jr  wrote:
>
> > Now, why would we *need* a 2.5.x branch?
>
> The primary need is to remove stuff that we deem not-ready-for-2.6-yet.
> Modules without any docs for example would need to be either documented or
> removed-from-2.5-that-will-become-2.6, keeping trunk as it is.


If it is not ready, it should not pollute trunk. Think of this as an
exercise to prune things unlikely to ever be hashed out completely. Why put
off the inevitable and force this set of issues on the 2.7.x maintainers
down the road?

We have a sandbox; things that must still be proven up need to be moved
there and pruned sometime in the next 10 years, if then.

Re: 2.4.x and 2.6.x ... next steps

[Stefan Eissing]

> Am 12.09.2018 um 16:19 schrieb Graham Leggett :
> 
> On 12 Sep 2018, at 15:39, William A Rowe Jr  wrote:
> 
>> Now, why would we *need* a 2.5.x branch?
> 
> The primary need is to remove stuff that we deem not-ready-for-2.6-yet. 
> Modules without any docs for example would need to be either documented or 
> removed-from-2.5-that-will-become-2.6, keeping trunk as it is.
> 
> We don’t want to remove work from trunk that needs polishing for the reasons 
> you’ve already described.

While we certainly do not want to dis any work in progress - and I think no one 
has that intention - trunk needs to be clean and ready for beta releases. 
Otherwise, it is just more work for all contributors. Someone who gets stuck in 
development of a feature should move it to its own feature branch for that very 
reason.

I feel myself in agreement with Bill that trunk needs to be where 2.5.x is born.

-Stefan

Re: 2.4.x and 2.6.x ... next steps

[William A Rowe Jr]

Going to largely ignore most other input on this thread, beyond the
underlying proposals to branch 2.5.x and move to RTC...

On Wed, Sep 12, 2018, 03:20 Stefan Eissing 
wrote:

> In my estimation, cleaning up trunk (or a copy of it) via RTC will take
> forever, at least.
>
> And while that continues, any bugfix can only be done in trunk. We then
> need 2(!) RTC proposals and votes per fix if it affects 2.4.x. (Which,
> until 2.6 is out and gets adopted, will be the case almost all of the time.)
>
> We do not even find enough people to look at the proposals for 2.4.x. It's
> easier to find people outside the project to test fixes in their production
> systems.
>
> Short: I  do not believe this can work.
>

I completely agree.

If we pause to consider how we got to http2 already, it was because of
Stefan's massive efforts *in CTR mode*. Would this have been a reasonable
project to launch in RTC? IMO, no.

If we trust our committers, CTR will let us make far more progress in far
less time. Note that all major proposals must be taken to the dev list
anyways.

Note that RTC fails to intercept as many bugs as it catches, as easily
observed during the 2.4.x generation, but at a very large cost in terms of
development feedback loop timing.

If we do not trust our committers we have a much different issue that RTC
does not address.

2.5.x are entirely alpha/beta releases, nobody will flinch at a regression
or bug.

Re: 2.4.x and 2.6.x ... next steps

[Jim Jagielski]

> On Sep 12, 2018, at 9:39 AM, William A Rowe Jr  wrote:
> 
> 
> So although others mentioned 2.4.x branch, this is not the origin of YOUR 
> proposal. Wow, that simplifies this discussion a lot (and hopefully, our new 
> committers who never even corresponded with some long absent colleagues now 
> see my concern with the dismissiveness against using trunk.) Sorry that I 
> misattributed that part of the discussion to your proposal.
> 
> Here is the problem I have with forking today. I expect you know I have no 
> problem with tagging 2.5.1-alpha any day of the week and putting 2.5 
> candidates up for release vote.
> 
> During the 'development' of 2.odd we have no need to fork, because all 
> API/ABI changes are permitted between 2.5.1 and 2.5.2. Our trunk needs to be 
> usable all the time, not just during this release prep. But, forking 
> introduces a mess of svn maintenance to no justified purpose.
> 
> And most of all, we need to trust our fellow committers. It is clear that 
> review before or after does not eliminate all errors. But 2.5 will not be GA 
> (2.6 will be.)
> 
> The straight line, avoiding a ton of excess backports, and keeping it simple 
> on the RM, is to either 1. Tag from the final, accepted 2.5.x svn commit rev 
> into 2.6.x branch, or 2. from trunk to 2.6.x branch if the RM believes the 
> changes are limited risk, and can be vetted during the release vote.
> 
> Forking 2.5.x says, outright, I don't trust my fellow committers with commit 
> bits to the alpha/beta development branch. That is also a bad signal to send 

Not at all. It sez "here is a safe space to continue to go wild and here is a 
place where there is an expected direct path to a releasable version"... no 
diff from when we had httpd-2.2, httpd-2.4 and trunk. We just have httpd-2.4, 
httpd-2.5 and trunk.

And again, ALL of this is predicated on the assumption that there is stuff in 
trunk (really useful and tested stuff) that cannot be backported to 2.4. 
Obviously, it is a lot easier, and takes a lot less time, to continue as we are 
and backport as we can; even some of the "impossible" stuff may actually be 
backportable with less effort than spinning 2.5/2.6. And, again personally, I 
think we are doing an extremely good job in balancing things between trunk and 
httpd-2.4, with most divergences either fluff or sandbox-type refactorings that 
may not pass muster when all is said and done.

Re: 2.4.x and 2.6.x ... next steps

[Graham Leggett]

On 12 Sep 2018, at 15:39, William A Rowe Jr  wrote:

> Now, why would we *need* a 2.5.x branch?

The primary need is to remove stuff that we deem not-ready-for-2.6-yet. Modules 
without any docs for example would need to be either documented or 
removed-from-2.5-that-will-become-2.6, keeping trunk as it is.

We don’t want to remove work from trunk that needs polishing for the reasons 
you’ve already described.

Regards,
Graham
—

Re: Cool Stuff In trunk: (Was: Re: 2.4.x and 2.6.x ... next steps)

[William A Rowe Jr]

On Wed, Sep 12, 2018 at 7:49 AM Jim Jagielski  wrote:

> As I said before, the main assumption in my suggestion is that there are
> things in trunk that "really should" be in some releasable version


Everything in trunk is now digested into three groups of commits, for
inspection. These don't include whitespace changes, so the resulting files
may be mis-indented and otherwise skewed, but we are looking for changes,
not insignificant formatting;

Just the docs (sources, and generated)
http://svn.apache.org/viewvc?view=revision=1840682
http://svn.apache.org/viewvc?view=revision=1840684

Just win32 generated build gunk (presuming vcproj and CMake are both
winners, there is a discussion to be had on .mak files before this gap is
addressed)
http://svn.apache.org/viewvc?view=revision=1840687

Everything else (all the interesting stuff)
http://svn.apache.org/viewvc?view=revision=1840688

I expect committers will be mostly interested in the last link above.

Re: 2.4.x and 2.6.x ... next steps

[William A Rowe Jr]

On Wed, Sep 12, 2018, 07:41 Jim Jagielski  wrote:

> Ahh. I think I see the problem! For some reason Bill sees this as somehow
> Jim's (my) fork. It's not. It's a  svn branch from HEAD of trunk, which
> contains
> all the changes. That branch is the projects's branch not some personal
> fork, whatever that means.
>

So although others mentioned 2.4.x branch, this is not the origin of YOUR
proposal. Wow, that simplifies this discussion a lot (and hopefully, our
new committers who never even corresponded with some long absent colleagues
now see my concern with the dismissiveness against using trunk.) Sorry that
I misattributed that part of the discussion to your proposal.

Here is the problem I have with forking today. I expect you know I have no
problem with tagging 2.5.1-alpha any day of the week and putting 2.5
candidates up for release vote.

During the 'development' of 2.odd we have no need to fork, because all
API/ABI changes are permitted between 2.5.1 and 2.5.2. Our trunk needs to
be usable all the time, not just during this release prep. But, forking
introduces a mess of svn maintenance to no justified purpose.

And most of all, we need to trust our fellow committers. It is clear that
review before or after does not eliminate all errors. But 2.5 will not be
GA (2.6 will be.)

The straight line, avoiding a ton of excess backports, and keeping it
simple on the RM, is to either 1. Tag from the final, accepted 2.5.x svn
commit rev into 2.6.x branch, or 2. from trunk to 2.6.x branch if the RM
believes the changes are limited risk, and can be vetted during the release
vote.

Forking 2.5.x says, outright, I don't trust my fellow committers with
commit bits to the alpha/beta development branch. That is also a bad signal
to send

Now, why would we *need* a 2.5.x branch? The only thing I can picture is
someone proposes post-2.6 work. And for the time being, such work can and
should live in a sandbox, imo.

Again, sorry I missed the transition from discussing trunk to discussing
2.4.x branch. I completely support your initial suggestion of the basis,
modulo tagging these pre-2.6.0 candidates directly on trunk.

Re: TLSv1.3 supprt for 2.4.x?

[Joe Orton]

On Tue, Sep 11, 2018 at 03:39:42PM +0200, Yann Ylavic wrote:
> On Tue, Sep 11, 2018 at 12:13 PM Joe Orton  wrote:
> >
> > Does anybody have successful test results with post-handshake auth?  I'm
> > testing against Fedora's OpenSSL 1.1.1pre9 which has merged the changes
> > for https://github.com/openssl/openssl/issues/6933
> 
> Just tried trunk+openssl-1.1.1pre9 (SSLProtocol TLSv1.3 and
> SSLVerifyClient require), with both firefox and s_client (w/ and w/o
> -enable_pha) and can't reproduce the hang.
> 
> What's your client tooling?

Have you tried the test suite with trunk+pre9?  I haven't had time to 
test this directly, but with Fedora 29's pre9 + tls1.3-2.4.x branch I am 
down to:

$ ./t/TEST t/ssl/
...
Failed 5/14 test programs. 12/331 subtests failed.

needed to get this far:

1. disable AUTO_RETRY to fix PHA
https://github.com/openssl/openssl/issues/7178#issuecomment-420300988

2. too much code has been moved out of ssl_hook_Access(), the 
FakeBasicAuth code & requires checking needs to come back

3. perl client side updates needed in IO::Socket:SSL and Net::SSLeay to 
re-enable PHA because of https://github.com/openssl/openssl/issues/6933

I will commit (1) and (2) today/tomorrow. I think (3) is, um, bizarre, 
but I'm not sure whether it is simpler to fight OpenSSL API design 
decisions or change every SSL client in the world to cope with them, 
probably the latter.

Re: svn commit: r1840585 - in /httpd/httpd/trunk: docs/log-message-tags/next-number modules/ssl/ssl_engine_kernel.c

[Stefan Eissing]

> Am 11.09.2018 um 19:12 schrieb Joe Orton :
> 
> On Tue, Sep 11, 2018 at 06:35:17PM +0200, Yann Ylavic wrote:
>> On Tue, Sep 11, 2018 at 6:01 PM  wrote:
>>> 
>>> Author: jorton
>>> Date: Tue Sep 11 16:01:47 2018
>>> New Revision: 1840585
>>> 
>>> URL: http://svn.apache.org/viewvc?rev=1840585=rev
>>> Log:
>>> * modules/ssl/ssl_engine_kernel.c (ssl_hook_Access_modern): Fail with
>>>  403 if SSL_verify_client_post_handshake() fails, e.g. when the
>>>  TLS/1.3 client didn't send the Post-Handshake Authentication
>>>  extension.
>> 
>> There also seems to be some subtilities between SSL_VERIFY_CLIENT_ONCE
>> (which we use in ssl_hook_Access_modern) and
>> SSL_VERIFY_POST_HANDSHAKES (another openssl flag related to PHA). I'm
>> not sure to understand the docs for now...
> 
> Uh, I missed that. I'm not sure why _VERIFY_CLIENT_*ONCE* is set there 
> rather than just _VERIFY_CLIENT... Stefan?  This should restrict PHA to 
> once per connection, maybe that is sensible, not sure.

This part was not easy to understand in the SSL documentation. I did not
manage to setup proper cases in my test suite to really pin this down.

How much have your testings now proceeded? Yann reported interop with
firefox for him against trunk. Did you manage to track down your
problems? Something missing in the branch?

Cheers,

Stefan

> 
>> Both seem to be mutually exclusive (though it's not really stated in
>> the doc), and possibly we don't use the right one since we call
>> SSL_verify_client_post_handshake() explicitely. On the other hand
>> SSL_VERIFY_POST_HANDSHAKES might depend on the client being PHA aware
>> (and/or advertised?), and if so should we detect it on the server side
>> to use SSL_VERIFY_POST_HANDSHAKES for the handshake?
>> 
>> I'm asking, should you have more insight on those flags...
> 
> I couldn't work out why SSL_VERIFY_POST_HANDSHAKE exists, but it didn't 
> seem to make any difference in testing here.  Assumed I was being stupid 
> but I've asked in https://github.com/openssl/openssl/issues/7178 now.
> 
> Regards, Joe

Re: Cool Stuff In trunk: (Was: Re: 2.4.x and 2.6.x ... next steps)

[Stefan Eissing]

> Am 12.09.2018 um 14:48 schrieb Jim Jagielski :
> 
> As I said before, the main assumption in my suggestion is that there are 
> things in trunk that "really should" be in some releasable version, stuff 
> significant enough to warrant the work, but is "impossible" to be backported 
> to 2.4.
> 
> If there are no real significant-but-impossible-to-backport features in 
> trunk, then the proposal itself is moot.
> 
> So let's think about it: What is currently in trunk that is a pretty 
> significant improvement? Then ask if it is directly backportable. Certainly 
> the effort in backporting from trunk to 2.4.x is much less than the effort in 
> spinning out 2.6.x and considering all things, that should be the primary 
> flow.

There are things I'd like to do for 2.5.x-to-become-2.6 releases that I cannot 
to in 2.4.x and will not do before that. I assume this holds also true for 
others.

To put it another way: current trunk is dead code to me. Only a stopover for 
2.4.x (aka release version). For the last three years, it was just in the way.

Or another way: I am too old to commit to trunk only. ;)

-Stefan

Cool Stuff In trunk: (Was: Re: 2.4.x and 2.6.x ... next steps)

[Jim Jagielski]

As I said before, the main assumption in my suggestion is that there are things 
in trunk that "really should" be in some releasable version, stuff significant 
enough to warrant the work, but is "impossible" to be backported to 2.4.

If there are no real significant-but-impossible-to-backport features in trunk, 
then the proposal itself is moot.

So let's think about it: What is currently in trunk that is a pretty 
significant improvement? Then ask if it is directly backportable. Certainly the 
effort in backporting from trunk to 2.4.x is much less than the effort in 
spinning out 2.6.x and considering all things, that should be the primary flow.

Re: 2.4.x and 2.6.x ... next steps

[Jim Jagielski]

Ahh. I think I see the problem! For some reason Bill sees this as somehow
Jim's (my) fork. It's not. It's a  svn branch from HEAD of trunk, which contains
all the changes. That branch is the projects's branch not some personal
fork, whatever that means.

> On Sep 12, 2018, at 6:49 AM, William A Rowe Jr  wrote:
> 
> On Wed, Sep 12, 2018, 04:16 Daniel Gruno  > wrote:
> On 09/12/2018 10:58 AM, Graham Leggett wrote:
> > On 12 Sep 2018, at 03:15, William A Rowe Jr  >  
> > >> wrote:
> > 
> >> On Tue, Sep 11, 2018, 17:42 Graham Leggett  >>  
> >> >> wrote:
> >>
> >> On 11 Sep 2018, at 20:35, Jim Jagielski  >> >> wrote:
> >>
> >> > This is what I propose:
> >> >
> >> >  o Later on this week I svn cp trunk over to branches/2.5.x
> >>
> >>
> >> -1 ... Veto on the basis of project bylaws. Propose a revision for voting.
> > 
> > I've totally lost you. Jim describes creating a branch, how is this 
> > related to voting?
> 
> I am not Bill, but it is likely a reference to the fact that you can 
> veto code changes, not community/workflow changes. I see Jim's proposal 
> as the latter, so I'm not sure why the attempted veto. The codebase 
> itself isn't being changed from what I can gather, only the workflow is.
> 
> Yes. To be clear, the proposal to make 2.5 Jim's fork, discarding all 
> previously committed changes to 2.5 (and I suppose, renumbering trunk as 2.7) 
> is a change to the project development process at httpd.
> 
> As-it-stands, such a personal fork is vetoable. And flies in sharp contrast 
> to community over code, discarding the existing collaboration of 2.5.1-dev 
> trunk in favor of one participants agenda.
> 
> Note, sandboxes are encouraged, don't require any vote to start one, and 
> might lead to some compromise between points a and b.
> 
> I suggest above, propose a *process* revision to our /dev/ docs for a vote, 
> before proceeding to violate community norms on versioning. Sorry for the 
> confusion about what I was proposing to 'revise'.
> 
> As a project committee vote to change our operational process, that is a 
> simple majority vote, as Daniel observes, which would carve out space for a 
> zombie (never to be released) trunk,

Re: 2.4.x and 2.6.x ... next steps

[Jim Jagielski]

> On Sep 11, 2018, at 4:57 PM, Eric Covener  wrote:
> 
>> + clearly document the changes in 2.4 -> 2.5/2.6, to start building the
>> next https://httpd.apache.org/docs/current/upgrading.html.
> 
> as well as docs/manual/new_features_2.5.xml
> 
> I am not sure 2.6 has much to offer. But it's a bit of a
> chicken-and-egg problem I guess.

My thought was that the suspend stuff and async filters were
significant/useful enough and that they were un-backportable,
hence what I wrote in the last paragraph that that improvement
may be the only real diff between 2.4 and 2.6.

Re: 2.4.x and 2.6.x ... next steps

[Eric Covener]

> Yes. To be clear, the proposal to make 2.5 Jim's fork, discarding all 
> previously committed changes to 2.5 (and I suppose, renumbering trunk as 2.7) 
> is a change to the project development process at httpd.

What's being discarded in the proposal?

Re: 2.4.x and 2.6.x ... next steps

[William A Rowe Jr]

On Wed, Sep 12, 2018, 04:16 Daniel Gruno  wrote:

> On 09/12/2018 10:58 AM, Graham Leggett wrote:
> > On 12 Sep 2018, at 03:15, William A Rowe Jr  > > wrote:
> >
> >> On Tue, Sep 11, 2018, 17:42 Graham Leggett  >> > wrote:
> >>
> >> On 11 Sep 2018, at 20:35, Jim Jagielski  >> > wrote:
> >>
> >> > This is what I propose:
> >> >
> >> >  o Later on this week I svn cp trunk over to branches/2.5.x
> >>
> >>
> >> -1 ... Veto on the basis of project bylaws. Propose a revision for
> voting.
> >
> > I've totally lost you. Jim describes creating a branch, how is this
> > related to voting?
>
> I am not Bill, but it is likely a reference to the fact that you can
> veto code changes, not community/workflow changes. I see Jim's proposal
> as the latter, so I'm not sure why the attempted veto. The codebase
> itself isn't being changed from what I can gather, only the workflow is.
>

Yes. To be clear, the proposal to make 2.5 Jim's fork, discarding all
previously committed changes to 2.5 (and I suppose, renumbering trunk as
2.7) is a change to the project development process at httpd.

As-it-stands, such a personal fork is vetoable. And flies in sharp contrast
to community over code, discarding the existing collaboration of 2.5.1-dev
trunk in favor of one participants agenda.

Note, sandboxes are encouraged, don't require any vote to start one, and
might lead to some compromise between points a and b.

I suggest above, propose a *process* revision to our /dev/ docs for a vote,
before proceeding to violate community norms on versioning. Sorry for the
confusion about what I was proposing to 'revise'.

As a project committee vote to change our operational process, that is a
simple majority vote, as Daniel observes, which would carve out space for a
zombie (never to be released) trunk,

Re: 2.4.x and 2.6.x ... next steps

[Daniel Gruno]

On 09/12/2018 10:58 AM, Graham Leggett wrote:
On 12 Sep 2018, at 03:15, William A Rowe Jr > wrote:


On Tue, Sep 11, 2018, 17:42 Graham Leggett > wrote:


On 11 Sep 2018, at 20:35, Jim Jagielski mailto:j...@jagunet.com>> wrote:

> This is what I propose:
>
>  o Later on this week I svn cp trunk over to branches/2.5.x


-1 ... Veto on the basis of project bylaws. Propose a revision for voting.


I've totally lost you. Jim describes creating a branch, how is this 
related to voting?


I am not Bill, but it is likely a reference to the fact that you can 
veto code changes, not community/workflow changes. I see Jim's proposal 
as the latter, so I'm not sure why the attempted veto. The codebase 
itself isn't being changed from what I can gather, only the workflow is.


With regards,
Daniel.



Rather than a wall of text, can you propose corrections to the above?

This is an attempt to discard the work of all committers who were told 
their code wouldn't be included until the next version major.minor. 
Complete disenfranchisement via a pocket veto of all changes.


Again, totally lost you. I can’t see anything in Jim’s proposal that 
suggests we throw away code or work from trunk, and I would not allow 
that to happen either. Have you mixed up the messages you’ve replied to?


Regards,
Graham
—

AW: 2.4.x and 2.6.x ... next steps

[Plüm , Rüdiger , Vodafone Group]

Von: William A Rowe Jr 
Gesendet: Mittwoch, 12. September 2018 03:15
An: httpd 
Betreff: Re: 2.4.x and 2.6.x ... next steps

On Tue, Sep 11, 2018, 17:42 Graham Leggett 
mailto:minf...@sharp.fm>> wrote:
On 11 Sep 2018, at 20:35, Jim Jagielski 
mailto:j...@jagunet.com>> wrote:



We have never dropped code without requiring an actual veto, or the committee 
withdrawing their code.

To my understanding we talk about code that “does not work” and has “issues”. 
So I have never seen a need to veto code in order to fix it. Fixing code can 
also include dropping parts of the code.
I guess you are more worried that we drop the code of complete “features” from 
trunk. As with other stuff I think we just need to discuss on dev@ if there are 
“features” that have issues that need to be fixed.
And if nobody pops up that wants or can fix these issues then dropping it is 
IMHO a valid option as well. If you want to do that via a formal veto on the 
original commit is another topic. I guess this is warranted for big
contributions like whole modules. I think that if we as a group decide in the 
open that a certain feature for a new major release is undesirable for whatever 
reason it is fine to drop it and thus the code even from trunk if nobody is 
willing to work on it and maintain it.
Actually the code will never be dropped. You just need to dig in the svn 
history to find it.
I think the whole point goes more in the direction that the review on trunk is 
sometimes too “sloopy” so that we accumulate unfinished, buggy and untested 
stuff instead of having these issues solved when the code was committed
and the contributor is still available and up to speed for discussion and 
resolution.
One other approach would point in the direction to have major releases more 
often such that these issues are detected earlier.
As proposed on this thread I guess a traversal of the stalled backport 
proposals for 2.4.x is a good starting point to pick up issues in trunk.
I guess we need to pay some price for getting 2.5.x / trunk in a GA releasable 
state in order to publish 2.6.x:


1.   Either we try to stabilize stuff on trunk, but this means we need to 
back down with new fancy ‘unstable’ commits on trunk (social rule not formal 
RTC).

2.   Or we add that 2.5.x branch which is RTC in between trunk and 2.4.x 
branch and thus slow down the backports to 2.4.

If there are enough people that can work on getting 2.5.x / trunk in a GA 
releasable state I think 1. or 2.  will not take long and thus are acceptable.
OTOH if doing that will become an endless story I think both approaches will 
create a lot of friction. Independent on which way we go we should possibly set 
a rough timeline for finishing this work and return to current state if it 
cannot be done this way in the timeline.

Regards

Rüdiger

Re: 2.4.x and 2.6.x ... next steps

[Graham Leggett]

On 12 Sep 2018, at 03:15, William A Rowe Jr  wrote:

> On Tue, Sep 11, 2018, 17:42 Graham Leggett  > wrote:
> On 11 Sep 2018, at 20:35, Jim Jagielski  wrote:
> 
> > This is what I propose:
> > 
> >  o Later on this week I svn cp trunk over to branches/2.5.x
> 
> -1 ... Veto on the basis of project bylaws. Propose a revision for voting.

I've totally lost you. Jim describes creating a branch, how is this related to 
voting?

Rather than a wall of text, can you propose corrections to the above?

> This is an attempt to discard the work of all committers who were told their 
> code wouldn't be included until the next version major.minor. Complete 
> disenfranchisement via a pocket veto of all changes.

Again, totally lost you. I can’t see anything in Jim’s proposal that suggests 
we throw away code or work from trunk, and I would not allow that to happen 
either. Have you mixed up the messages you’ve replied to?

Regards,
Graham
—

Re: 2.4.x and 2.6.x ... next steps

[Stefan Eissing]

In my estimation, cleaning up trunk (or a copy of it) via RTC will take 
forever, at least.

And while that continues, any bugfix can only be done in trunk. We then need 
2(!) RTC proposals and votes per fix if it affects 2.4.x. (Which, until 2.6 is 
out and gets adopted, will be the case almost all of the time.)

We do not even find enough people to look at the proposals for 2.4.x. It's 
easier to find people outside the project to test fixes in their production 
systems. 

Short: I do not believe this can work.

-Stefan

> Am 11.09.2018 um 20:35 schrieb Jim Jagielski :
> 
> I'd like for us to seriously consider the next steps
> related to the future of httpd.
> 
> In trunk we have some stuff that can be easily, or, at
> least, *somewhat* easily backported to 2.4.x, and I
> personally think that we should do that. But we also
> have some items which cannot be backported due to API/ABI
> concerns, and some of these are pretty useful things.
> And finally, we have some things in trunk that will likely
> need to be majorly fixed or else scrapped.
> 
> The 1st thing we need to do is classify those things
> within trunk. We then need to backport what we can,
> and should, and then make progress on a 2.6.x release
> (please note, I am using shorthand here... yes, I know
> what we *really* do is a 2.5.x which then goes to 2.6.x
> but I'm hopeful we all know what I mean).
> 
> This is what I propose:
> 
>  o Later on this week I svn cp trunk over to branches/2.5.x
>  o That branch becomes the initial source for 2.6.x
>  o trunk remains CTR, whereas branches/2.5.x becomes RTC
>ala 2.4.x (ie: using STATUS and specific, targeted
>backport requests).
>  o Backports to 2.4.x only come from 2.5.x
>  o We then release a 2nd alpha from branches/2.5.x
>  o We get 2.5.x into a releasable stage, whereas we
>svn mv branches/2.5.x to branches/2.6.x
> 
> A combination of trunk's STATUS and 2.4.x's STATUS will
> become the STATUS for 2.5.x... see below for the why (but
> basically that file will serve as the place where those
> above "classifications" will be documented).
> 
> The main goal is that this creates a somewhat "stable"
> 2.5.x branch which can be polished but as well serve as
> the source for backports. Additional development continues
> on trunk w/o mussing up 2.5.x... but there is also a path
> that stuff in trunk can end-up in 2.6.x. In also allows us
> to remove "experiments" in the old trunk (and now 2.5.x)
> which are broken or, at least, doesn't have enough support
> to warrant being released (a glance thru 2.4.x's STATUS file
> for backports which are stagnated, etc provide some clues
> on what those could be... at the least, this will provide
> incentive to address those concerns or revert those additions)
> 
> For me, the main push for this are some of the various async
> improvements in trunk that, at least from what I can tell,
> simply cannot be backported. It is possible that those improvements
> will be the primary and almost-singular distinction between
> 2.4.x and 2.6.x after all is said and done, but who knows...
> 
> Thoughts? Comments?