Implement hot spares in mod_proxy_balancer [was Re: Ideas from ApacheCon]

[Jim Riggs]

> On 1 Jun 2017, at 17:15, Yann Ylavic  wrote:
> 
> On Thu, Jun 1, 2017 at 10:48 PM, Jim Riggs  wrote:
>>> On 1 Jun 2017, at 15:25, Yann Ylavic  wrote:
>>> 
>>> On Thu, Jun 1, 2017 at 10:22 PM, Yann Ylavic 
>>> wrote:
 On Thu, Jun 1, 2017 at 7:29 PM, Jim Riggs 
 wrote:
>> On 1 Jun 2017, at 07:55, Jim Jagielski 
>> wrote: 2. I understand the logic behind creating the arrays,
>> but doesn't this increase the overhead. We go thru the full
>> list of workers one time, and then go thru the list of avail
>> works and spares right after that. Assume that all workers
>> are available; doesn't it mean we go thru that last 2x?
 []
> 
> The only way I can think of to avoid this without going back
> to duplicating code across lbmethods, which I would argue is
> worse, is to maybe have the lbmethod provide a callback
> function and context pointer to
> ap_proxy_balancer_usable_workers() that gets called during the
> loop iteration to pick the best member in flight.
 
 Couldn't a simple 'best' for ap_proxy_balancer_usable_workers()
 make it return a single entry?
>>> 
>>> ... a simple 'best' *flag* (as argument) ...
>> 
>> I'm not sure I follow what this flag would be. The lbmethod would
>> somehow have to tell ap_proxy_balancer_usable_workers() how to pick
>> the best worker (e.g. by comparing the number of bytes sent or the
>> number of requests processed). I'm not sure how that information
>> could be passed as a flag unless we baked the behavior of byrequests,
>> bybusyness, and bytraffic into ap_proxy_balancer_usable_workers().
>> But then how would we allow for plugging in additional lbmethods?
> 
> Oh right, nevermind, I thought per lbmethod logic was already there.
> After a better look into it, a callback (and its baton) with that
> logic looks straightforward, though.

Indeed. I put a new patch that uses callbacks on BZ61140 (and visually 
available at https://github.com/jhriggs/httpd/pull/1/files). So, the lbmethods 
call ap_proxy_balancer_get_best_worker() with a callback and baton so that the 
best worker is chosen in flight during iteration of the workers, addressing 
Jim's overhead concern. Changes:

- use a callback and baton to the lbmethod in
  ap_proxy_balancer_get_best_worker() (previously
  ap_proxy_balancer_usable_workers()) so that we can find the best worker in
  flight while iterating rather than returning an array of usable workers that
  has to then in turn be iterated

- consider a draining worker unusable and suitable for replacement by a spare


> Regarding:
> worker = _ARRAY_IDX(balancer->workers, i, proxy_worker *);
> I don't see why worker needs to be a 'proxy_worker**' ('*worker' is
> never updated IIUC).
> 
> Looks like:
> proxy_worker *worker = APR_ARRAY_IDX(balancer->workers, i, proxy_worker *);
> would be fine and allow using 'worker->xxx' instead of
> '(*worker)->xxx' all over the place...

Much of what I did was based on what already existed. The existing code was 
iterating through proxy_worker**, so mine did too. You are correct, though, 
that there is no real reason for it. The latest patch switches to just using * 
rather than **.

Thanks for putting eyes on this! I appreciate it.

Re: HTTP/2 and no-longer "experimental"

[Jacob Champion]

On 05/31/2017 08:11 AM, Graham Leggett wrote:

+1 to no-longer-experimental.

+1 to RTC.


+1 to both.

--Jacob

Re: HTTP/2 and no-longer "experimental"

[Yann Ylavic]

On Wed, May 31, 2017 at 2:23 PM, Eric Covener  wrote:
> On Wed, May 31, 2017 at 8:07 AM, Jim Jagielski  wrote:
>> There was discussion some time ago about dropping the "experimental"
>> tag from our HTTP/2 implementation. It is causing loads of people
>> to not use it, as well as allowing for the perpetuation of FUD that
>> httpd really doesn't support HTTP/2.
>>
>> I'd like for 2.4.26 to be the release that removes that tag. It
>> implies a transition to RTC in 2.4 for it, but I think that that
>> is workable and realistic at this point...
>>
>> Thoughts? Comments?
>
> I think we should drop the experimental label/treatment. If not now,
> it seems like it's effectively permanent in 2.4.

+1

Re: Ideas from ApacheCon

[Yann Ylavic]

On Thu, Jun 1, 2017 at 10:48 PM, Jim Riggs  wrote:
>> On 1 Jun 2017, at 15:25, Yann Ylavic  wrote:
>>
>> On Thu, Jun 1, 2017 at 10:22 PM, Yann Ylavic 
>> wrote:
>>> On Thu, Jun 1, 2017 at 7:29 PM, Jim Riggs 
>>> wrote:
> On 1 Jun 2017, at 07:55, Jim Jagielski 
> wrote: 2. I understand the logic behind creating the arrays,
> but doesn't this increase the overhead. We go thru the full
> list of workers one time, and then go thru the list of avail
> works and spares right after that. Assume that all workers
> are available; doesn't it mean we go thru that last 2x?
>>> []

 The only way I can think of to avoid this without going back
 to duplicating code across lbmethods, which I would argue is
 worse, is to maybe have the lbmethod provide a callback
 function and context pointer to
 ap_proxy_balancer_usable_workers() that gets called during the
 loop iteration to pick the best member in flight.
>>>
>>> Couldn't a simple 'best' for ap_proxy_balancer_usable_workers()
>>> make it return a single entry?
>>
>> ... a simple 'best' *flag* (as argument) ...
>
> I'm not sure I follow what this flag would be. The lbmethod would
> somehow have to tell ap_proxy_balancer_usable_workers() how to pick
> the best worker (e.g. by comparing the number of bytes sent or the
> number of requests processed). I'm not sure how that information
> could be passed as a flag unless we baked the behavior of byrequests,
> bybusyness, and bytraffic into ap_proxy_balancer_usable_workers().
> But then how would we allow for plugging in additional lbmethods?

Oh right, nevermind, I thought per lbmethod logic was already there.
After a better look into it, a callback (and its baton) with that
logic looks straightforward, though.

Regarding:
 worker = _ARRAY_IDX(balancer->workers, i, proxy_worker *);
I don't see why worker needs to be a 'proxy_worker**' ('*worker' is
never updated IIUC).

Looks like:
 proxy_worker *worker = APR_ARRAY_IDX(balancer->workers, i, proxy_worker *);
would be fine and allow using 'worker->xxx' instead of
'(*worker)->xxx' all over the place...


Regards,
Yann.

Re: Ideas from ApacheCon

[Jim Riggs]

> On 1 Jun 2017, at 15:25, Yann Ylavic  wrote:
> 
> On Thu, Jun 1, 2017 at 10:22 PM, Yann Ylavic  wrote:
>> On Thu, Jun 1, 2017 at 7:29 PM, Jim Riggs  wrote:
 On 1 Jun 2017, at 07:55, Jim Jagielski  wrote:
 2. I understand the logic behind creating the arrays, but doesn't
this increase the overhead. We go thru the full list of workers
one time, and then go thru the list of avail works and spares
right after that. Assume that all workers are available; doesn't
it mean we go thru that last 2x?
>> []
>>> 
>>> The only way I can think of to avoid this without going back to
>>> duplicating code across lbmethods, which I would argue is worse, is
>>> to maybe have the lbmethod provide a callback function and context
>>> pointer to ap_proxy_balancer_usable_workers() that gets called during
>>> the loop iteration to pick the best member in flight.
>> 
>> Couldn't a simple 'best' for ap_proxy_balancer_usable_workers() make
>> it return a single entry?
> 
> ... a simple 'best' *flag* (as argument) ...

I'm not sure I follow what this flag would be. The lbmethod would somehow have 
to tell ap_proxy_balancer_usable_workers() how to pick the best worker (e.g. by 
comparing the number of bytes sent or the number of requests processed). I'm 
not sure how that information could be passed as a flag unless we baked the 
behavior of byrequests, bybusyness, and bytraffic into 
ap_proxy_balancer_usable_workers(). But then how would we allow for plugging in 
additional lbmethods?


>> 
>> By the way, retrieving ap_proxy_retry_worker() via an OPTIONAL_FN
>> looks unnecessary since it's implemented in the exact same
>> "proxy_util.c" file.

LOL! That's a copy-paste remnant from when I had that function in a different 
file originally. Patch updated. Thanks for catching it!

Re: Ideas from ApacheCon

[Yann Ylavic]

On Thu, Jun 1, 2017 at 10:22 PM, Yann Ylavic  wrote:
> On Thu, Jun 1, 2017 at 7:29 PM, Jim Riggs  wrote:
>>> On 1 Jun 2017, at 07:55, Jim Jagielski  wrote:
>>>
>>> I really like where this is going... I just have a few questions:
>>>
>>> 1. The style, esp with array usage is different; eg APR_ARRAY_PUSH
>>> and APR_ARRAY_IDX... any particular reason why?
>>
>> Well, we don't seem to be entirely consistent with using elts and
>> apr_array*() directly vs. the helper #defines across the codebase. I
>> was doing everything with direct pointer access and arithmetic until
>> I started looking at the heartbeat lbmethod that was using some of
>> the helpers. IMNSHO, I think they are a little cleaner and easier
>> use/read. I got lost in dereferencing hell with all of the `&((*...)
>> **)->(*)'! :-) The helpers are there, and I think they are convenient
>> and worth using, but I'm not digging through all of this code as
>> regularly as some of the others on here. If the standard/consensus is
>> the other way, it's easy to change.
>
> I also (much, much) prefer the macros, and preferably would like all
> the 'elts' usages and cast trickeries changed to them rather. But
> that's another (own) story...
>
>>
>>
>>>  2. I understand the logic behind creating the arrays, but doesn't
>>> this increase the overhead. We go thru the full list of workers
>>> one time, and then go thru the list of avail works and spares
>>> right after that. Assume that all workers are available; doesn't
>>> it mean we go thru that last 2x?
> []
>>
>> The only way I can think of to avoid this without going back to
>> duplicating code across lbmethods, which I would argue is worse, is
>> to maybe have the lbmethod provide a callback function and context
>> pointer to ap_proxy_balancer_usable_workers() that gets called during
>> the loop iteration to pick the best member in flight.
>
> Couldn't a simple 'best' for ap_proxy_balancer_usable_workers() make
> it return a single entry?

... a simple 'best' *flag* (as argument) ...

>
> By the way, retrieving ap_proxy_retry_worker() via an OPTIONAL_FN
> looks unnecessary since it's implemented in the exact same
> "proxy_util.c" file.
>
>>
>> Regardless, even worst case, we are looking at what, iterating 6
>> pointers instead of 3 or 10 instead of 5? We probably have some lower
>> hanging fruit across the request lifecycle code to increase
>> performance than saving some arithmetic on a handful of structs, no?
>> ;-)
>
> Agreed.

Re: Ideas from ApacheCon

[Yann Ylavic]

On Thu, Jun 1, 2017 at 7:29 PM, Jim Riggs  wrote:
>> On 1 Jun 2017, at 07:55, Jim Jagielski  wrote:
>>
>> I really like where this is going... I just have a few questions:
>>
>> 1. The style, esp with array usage is different; eg APR_ARRAY_PUSH
>> and APR_ARRAY_IDX... any particular reason why?
>
> Well, we don't seem to be entirely consistent with using elts and
> apr_array*() directly vs. the helper #defines across the codebase. I
> was doing everything with direct pointer access and arithmetic until
> I started looking at the heartbeat lbmethod that was using some of
> the helpers. IMNSHO, I think they are a little cleaner and easier
> use/read. I got lost in dereferencing hell with all of the `&((*...)
> **)->(*)'! :-) The helpers are there, and I think they are convenient
> and worth using, but I'm not digging through all of this code as
> regularly as some of the others on here. If the standard/consensus is
> the other way, it's easy to change.

I also (much, much) prefer the macros, and preferably would like all
the 'elts' usages and cast trickeries changed to them rather. But
that's another (own) story...

>
>
>>  2. I understand the logic behind creating the arrays, but doesn't
>> this increase the overhead. We go thru the full list of workers
>> one time, and then go thru the list of avail works and spares
>> right after that. Assume that all workers are available; doesn't
>> it mean we go thru that last 2x?
[]
>
> The only way I can think of to avoid this without going back to
> duplicating code across lbmethods, which I would argue is worse, is
> to maybe have the lbmethod provide a callback function and context
> pointer to ap_proxy_balancer_usable_workers() that gets called during
> the loop iteration to pick the best member in flight.

Couldn't a simple 'best' for ap_proxy_balancer_usable_workers() make
it return a single entry?

By the way, retrieving ap_proxy_retry_worker() via an OPTIONAL_FN
looks unnecessary since it's implemented in the exact same
"proxy_util.c" file.

>
> Regardless, even worst case, we are looking at what, iterating 6
> pointers instead of 3 or 10 instead of 5? We probably have some lower
> hanging fruit across the request lifecycle code to increase
> performance than saving some arithmetic on a handful of structs, no?
> ;-)

Agreed.

Re: Ideas from ApacheCon

[Jim Riggs]

> On 1 Jun 2017, at 07:55, Jim Jagielski  wrote:
> 
> I really like where this is going... I just have a few questions:
> 
>  1. The style, esp with array usage is different; eg APR_ARRAY_PUSH
> and APR_ARRAY_IDX... any particular reason why?

Well, we don't seem to be entirely consistent with using elts and apr_array*() 
directly vs. the helper #defines across the codebase. I was doing everything 
with direct pointer access and arithmetic until I started looking at the 
heartbeat lbmethod that was using some of the helpers. IMNSHO, I think they are 
a little cleaner and easier use/read. I got lost in dereferencing hell with all 
of the `&((*...) **)->(*)'! :-) The helpers are there, and I think they are 
convenient and worth using, but I'm not digging through all of this code as 
regularly as some of the others on here. If the standard/consensus is the other 
way, it's easy to change.


>  2. I understand the logic behind creating the arrays, but doesn't
> this increase the overhead. We go thru the full list of workers
> one time, and then go thru the list of avail works and spares
> right after that. Assume that all workers are available; doesn't
> it mean we go thru that last 2x?

If everything is available, the old implementation only goes through the list 
once and picks a worker. This new implementation goes through the list once and 
returns a *subset* that the lbmethod then has to loop through to pick a worker. 
Worst case (e.g. X workers all available in lbset 0), yes, we would now do 2X.

In a more complex scenario (e.g. X workers all available in each lbsets 0, 1, 
and 2), we do 3X (1st loop) + X (2nd loop) = 4X, not 2 * 3X = 6X. So, yes, 
there is slightly more overhead...but, the second loop (in the lbmethod) has at 
most one lbset's worth of workers and really isn't doing anything except an 
arithmetic comparison to pick the best from them.

If a number of workers are unavailable, however, it's different. The old 
implementation loops the whole list twice per lbset if it doesn't find a 
worker, once for workers and once for standbys, whereas this new implementation 
still loops the whole list only once per lbset. In the example above, 2 * 3X = 
6X for old, 3X + X = 4X for new.

The only way I can think of to avoid this without going back to duplicating 
code across lbmethods, which I would argue is worse, is to maybe have the 
lbmethod provide a callback function and context pointer to 
ap_proxy_balancer_usable_workers() that gets called during the loop iteration 
to pick the best member in flight. I am open to that...it just seemed to add 
unnecessary complexity.

Regardless, even worst case, we are looking at what, iterating 6 pointers 
instead of 3 or 10 instead of 5? We probably have some lower hanging fruit 
across the request lifecycle code to increase performance than saving some 
arithmetic on a handful of structs, no? ;-)


>> On May 31, 2017, at 1:44 PM, Jim Riggs  wrote:
>> 
>>> On 23 May 2017, at 09:16, Jim Riggs  wrote:
>>> 
 On 18 May 2017, at 13:22, Rainer Jung  wrote:
 
 Am 18.05.2017 um 19:46 schrieb Jim Jagielski:
> Based on feedback from various sessions:
> 
> o A new-kind of "hot standby" in mod_proxy which kicks
> in whenever a worker moves out of the pool (ie, doesn't
> wait until all workers are out)... ala a redundant
> hard drive.
 
 Maybe "spare worker" (and we could have more than one such spares).
>>> 
>>> Exactly. I am already working on some code for this, though it to seems to 
>>> necessarily be a bit convoluted in the current codebase.
>>> 
>>> The way we treat a "hot standby" in mod_proxy_balancer is as a last-ditch 
>>> effort to return something. I.e. *all* workers are unavailable, so then we 
>>> use the hot standby. (This problem can actually be solved a different way 
>>> by setting a high value for lbset.)
>>> 
>>> In my mind, though, what is proposed here is actually how I actually expect 
>>> a "hot standby" to work. Think of it more like a "hot spare" in disk RAID 
>>> terms. That is, if *any* worker is unavailable, the hot spare will be used 
>>> (or at least added to the list of potential workers...still to be 
>>> determined by the lbmethod implementation).
>>> 
>>> Example:
>>> 
>>> 
>>> BalancerMember "http://192.168.1.50:80;
>>> BalancerMember "http://192.168.1.51:80;
>>> BalancerMember "http://192.168.1.52:80;
>>> BalancerMember "http://192.168.1.53:80; status=+H
>>> 
>>> 
>>> In this case, .53 will only get used if .50, .51, and .52 are *all* 
>>> unavailable.
>>> 
>>> 
>>> BalancerMember "http://192.168.1.50:80;
>>> BalancerMember "http://192.168.1.51:80;
>>> BalancerMember "http://192.168.1.52:80;
>>> BalancerMember "http://192.168.1.53:80; status=+R # new "hot spare" status
>>> BalancerMember "http://192.168.1.54:80; status=+R # new "hot spare" status
>>> 
>>> 
>>> In this case, if .50 

Re: svn commit: r1707087 - /httpd/httpd/trunk/modules/debugging/mod_bucketeer.c

[Jacob Champion]

On 05/02/2017 10:14 AM, Ruediger Pluem wrote> On 04/28/2017 10:50 PM, 
Jacob Champion wrote:

...why does the EOR bucket have memory ownership of a request_rec,
especially when its lifetime is not well defined? And why have we
put business logic into a finalizer? This is ringing all of my
memory management alarm bells.


This dates back to a long time ago
(http://svn.apache.org/viewvc?view=revision=327925) when we
started to add async processing to httpd. We had the issue that we
could not just destroy the request pool, once we "finished" the 
processing of a request, because we could still have buckets and data

created out of the request pool in the async processing of the
filters. Hence the idea was to sent a special final request bucket as
the very last bucket of a request that tells the filter that consumes
the bucket (usually the core output filter) that no data for the
request is coming down the filter stack and that it is now save to
destroy the request pool. And in order to prevent that logic to be
coded in the filter (when it just sees the bucket) it was put in the
"finalizer" code of the eor bucket. You might find this strange, but
IMHO we have this pattern to use cleanups for some business logic
quite often in httpd and we tie a lot of this logic to the lifecycle
of pools. So pools are not just a memory management tool, but also a
lifecycle management tool.


Hi Rüdiger,

I just realized I hadn't responded to this; sorry! Thanks for the 
background.


I have been trying (slowly) to learn enough about the async processing 
chain to participate meaningfully in this discussion, but so far I 
haven't had the time to really dig in. My intuition is that, if pieces 
of the request are disappearing out from under the thread, something is 
wrong with the lifecycle dependency chain (and I've never seen finalizer 
dependencies work out very well in practice). But it's just an idle 
guess without more research.


AFAIK, this is one of the final blockers (if not *the* final blocker, 
fingers crossed) for the trunk autotesting system; hence my interest.


--Jacob

Re: Ideas from ApacheCon

[Jim Jagielski]

I really like where this is going... I just have a few questions:

  1. The style, esp with array usage is different; eg APR_ARRAY_PUSH
 and APR_ARRAY_IDX... any particular reason why?

  2. I understand the logic behind creating the arrays, but doesn't
 this increase the overhead. We go thru the full list of workers
 one time, and then go thru the list of avail works and spares
 right after that. Assume that all workers are available; doesn't
 it mean we go thru that last 2x?

> On May 31, 2017, at 1:44 PM, Jim Riggs  wrote:
> 
>> On 23 May 2017, at 09:16, Jim Riggs  wrote:
>> 
>>> On 18 May 2017, at 13:22, Rainer Jung  wrote:
>>> 
>>> Am 18.05.2017 um 19:46 schrieb Jim Jagielski:
 Based on feedback from various sessions:
 
 o A new-kind of "hot standby" in mod_proxy which kicks
 in whenever a worker moves out of the pool (ie, doesn't
 wait until all workers are out)... ala a redundant
 hard drive.
>>> 
>>> Maybe "spare worker" (and we could have more than one such spares).
>> 
>> Exactly. I am already working on some code for this, though it to seems to 
>> necessarily be a bit convoluted in the current codebase.
>> 
>> The way we treat a "hot standby" in mod_proxy_balancer is as a last-ditch 
>> effort to return something. I.e. *all* workers are unavailable, so then we 
>> use the hot standby. (This problem can actually be solved a different way by 
>> setting a high value for lbset.)
>> 
>> In my mind, though, what is proposed here is actually how I actually expect 
>> a "hot standby" to work. Think of it more like a "hot spare" in disk RAID 
>> terms. That is, if *any* worker is unavailable, the hot spare will be used 
>> (or at least added to the list of potential workers...still to be determined 
>> by the lbmethod implementation).
>> 
>> Example:
>> 
>> 
>> BalancerMember "http://192.168.1.50:80;
>> BalancerMember "http://192.168.1.51:80;
>> BalancerMember "http://192.168.1.52:80;
>> BalancerMember "http://192.168.1.53:80; status=+H
>> 
>> 
>> In this case, .53 will only get used if .50, .51, and .52 are *all* 
>> unavailable.
>> 
>> 
>> BalancerMember "http://192.168.1.50:80;
>> BalancerMember "http://192.168.1.51:80;
>> BalancerMember "http://192.168.1.52:80;
>> BalancerMember "http://192.168.1.53:80; status=+R # new "hot spare" status
>> BalancerMember "http://192.168.1.54:80; status=+R # new "hot spare" status
>> 
>> 
>> In this case, if .50 becomes unavailable, .53 (or .54 depending on 
>> implementation) will be treated as an available worker for the lbmethod to 
>> potentially choose. If 2 or more of .50, .51, and .52 become unavailable, 
>> both .53 and .54 would be available to be chosen.
>> 
>> So, instead of having a single fallback option when *all* workers are dead, 
>> we will have a way of trying to ensure that a specific number of workers (3 
>> in the example above) are always available...just like a hot spare drive 
>> plugs into the RAID array when one of the members dies. In our case, though, 
>> once the main worker recovers, the hot spare will go back to being a hot 
>> spare (except for matching routes).
>> 
>> Comments welcome.
> 
> 
> As promised, balancer hot spare support: 
> https://bz.apache.org/bugzilla/show_bug.cgi?id=61140
> 

Re: HTTP/2 and no-longer "experimental"

[Eric Covener]

On Thu, Jun 1, 2017 at 7:10 AM, Jim Jagielski  wrote:
>>
>> - B. People come out and say: "when experimental is gone, I might finally
>> find the time to improving HTTP/2 support with the great ideas and/or long
>> time experience that I have!"
>>
>
> There is that as well... it's no good having a substantial module
> with very, very limited expertise shared and involvement.

I don't really think developers are sidelined by this directly, but
maybe indirectly by the adoption reduced by A).

-- 
Eric Covener
cove...@gmail.com

Re: HTTP/2 and no-longer "experimental"

[Jim Jagielski]

> On Jun 1, 2017, at 4:40 AM, Stefan Eissing  
> wrote:
> 
> What could a change of "experimental" do?
> 
> - A. It could address the FUD. Which I assume is important for market shares. 
> And for people who have done serious investments (successful ones) in httpd 
> in the past, be it money or time.

Not so much market shares, for market share reasons, but simply
that people who could and should be using httpd for http/2 simply
aren't. It's a blocker to having more people use it, which means
less bug reports and less feedback on fixes, etc...

> 
> - B. People come out and say: "when experimental is gone, I might finally 
> find the time to improving HTTP/2 support with the great ideas and/or long 
> time experience that I have!"
> 

There is that as well... it's no good having a substantial module
with very, very limited expertise shared and involvement.

Re: HTTP/2 and no-longer "experimental"

[Stefan Eissing]

> Am 31.05.2017 um 17:46 schrieb William A Rowe Jr :
> 
> I have the impression that the developers still believe HTTP/2
> proxy is still 'experimental' / work-in-progress. Notably, there is
> a large pile of duplicate functionality in the modules/http2/ tree
> which should already be promoted to httpd util commons, so
> one copy of these duplicated functions are shared by both
> mod_http2 and mod_proxy_http2 (as well as potential http/2
> enhancement modules).

The developer of all this agrees that mod_proxy_http2 should stay experimental.
I would even recommend to remove mod_proxy_http2 from 2.4.x, until someone
finds time to address the problems reported. It works well for me, but some 
people
observe large transfers not working on Windows, for example.

> I have the impression that mod_http2 implementation in 2.6 is
> already cleaner and more maintainable, owing to enhancements
> Stefan already contributed and those parts of the implementation
> that httpd 2.4 had subpar support for... leading to various bits of
> bubblegum and twists of bailing wire, which are harder to maintain
> without the 2.6 API fixes.

FYI: I can also assure that code between trunk and 2.4.x is identical
except ap_create_request(), introduced in trunk, that was never back-ported.

My stance on mod_http2's experimental status is that I do not really
have an opinion. People seem to have more time and energy available, 
the farther away from actual code the discussion runs. Which is contrary 
to how I operate.

So:
> Am 31.05.2017 um 14:07 schrieb Jim Jagielski :
> 
> There was discussion some time ago about dropping the "experimental"
> tag from our HTTP/2 implementation. It is causing loads of people
> to not use it, as well as allowing for the perpetuation of FUD that
> httpd really doesn't support HTTP/2.

What could a change of "experimental" do?

- A. It could address the FUD. Which I assume is important for market shares. 
And for people who have done serious investments (successful ones) in httpd 
in the past, be it money or time.

- B. People come out and say: "when experimental is gone, I might finally 
find the time to improving HTTP/2 support with the great ideas and/or long 
time experience that I have!"


On A, I am not interested. I will neither promote nor disapprove any change 
there. I just does not matter to me, personally. But if you want to talk 
about B, I am all ears!

Cheers,

Stefan

PS. Merit: the following people, beside myself, have contributed to 
the HTTP/2 efforts (to the best of my and svn praise memory):

- Yann did much work in analyzing crashes and made fixes
- Jim did the original import and added conn_rec* master
- Graham (minfrin) added ap_create_request() to trunk
- jfclere and jchampion did some code formatting/cleanups
- rjung added forgotten APLOGNOs
- jailletc and elukey worked a lot on the docs
- many, many people tested

Thank you!