Re: Start using RCs (Was: Re: So... when should we do 2.4.34? [WAS: Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)])

[William A Rowe Jr]

On Tue, Apr 24, 2018 at 6:36 AM, Daniel Ruggeri  wrote:
>
> One more thing to point out that I didn't explicitly say in the previous 
> message is that this suggestion implies the release branch regularly gets cut 
> from trunk (rather than growing and diverging on its own). This helps avoid 
> "locking" features in trunk indefinitely because of the time between Maj.Min 
> bumps.

+1... the new development branch has the greatest activity level. Any patch
branch is picked from that current activity.

Any major rev refactoring should be proven up in a sandbox first. If it can
be automated (e.g. function renames or mass function updates as we had
for APLOGNO()), all the better to test and re-test against trunk.

Re: Start using RCs (Was: Re: So... when should we do 2.4.34? [WAS: Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)])

[Daniel Ruggeri]

On April 23, 2018 11:30:07 AM CDT, William A Rowe Jr  
wrote:
>On Sun, Apr 22, 2018 at 11:34 AM, Daniel Ruggeri 
>wrote:
>>
>> The more I think about it, the more I *really* like a semver-ish
>> approach where major represents the ABI that will not be broken,
>minor
>> represents the feature set (for backwards compatibility) and patch
>> represents forward-compatible changes/fixes. This may be, in spirit,
>> what you are suggesting. I'd further add that no directives are added
>in
>> patch releases, but minor releases would be fair game. How we choose
>to
>> maintain that behind the scenes is a thought exercise. Given that
>most
>> of our users get httpd from distros, I'd be in favor of having only
>two
>> long-lived branches: trunk and release. The distros are already
>taking
>> the point-release they want and curating fixes/changes for it, so
>they
>> will be doing what they do regardless of our direction.
>
>Thought exercise... if it is decided that version 3.5.x added an all
>around
>nice set of features, and we had platform maintainers as committers who
>wanted to keep backporting to that tree at this project, well... they
>could
>elect to maintain that version as a long lived branch and collaborate
>on
>it while we went on our merry way through 3.15.0, even 4.1.x as a
>group.
>
>In other words, wherever we have three committers who will cultivate a
>long lived release, they can do that. This doesn't differ from how a
>few
>individuals kept 2.0, and later, 2.2 alive for a significant period of
>time
>after the next major release. When interest cooled, each was retired.

Of course - this being Open Source and we being volunteers, we work on what we 
want to work on :-). I generally see that doesn't happen much for projects 
operating this way, though. My perception is that typically once 2.7.0 is 
released, work dies off quickly for 2.6.x. As mentioned elsethread, what the 
distros are signing up for (right or wrong) is to put a stake in the ground on 
a certain version and to select specific patches for fixes alone. If we were to 
do the same, it may be a duplication of effort.

Now... That doesn't preclude us from operating like Ubuntu does, for example, 
with their LTS style releases where we do "2.6.x will get fixes only for X 
years, but 2.7+ will get features until the next LTS cut". I think that idea 
may be what you have in mind. So, in that world we have three long lived 
branches: trunk, release and LTS. I can see us operating in that way... and to 
make backports less cumbersome, we could further declare that a fix accepted in 
STATUS for the LTS branch is accepted also into the release branch.

One more thing to point out that I didn't explicitly say in the previous 
message is that this suggestion implies the release branch regularly gets cut 
from trunk (rather than growing and diverging on its own). This helps avoid 
"locking" features in trunk indefinitely because of the time between Maj.Min 
bumps.
-- 
Daniel Ruggeri

AW: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)

[Plüm , Rüdiger , Vodafone Group]

> -Ursprüngliche Nachricht-
> Von: David Zuelke <dzue...@salesforce.com>
> Gesendet: Montag, 23. April 2018 18:09
> An: dev@httpd.apache.org
> Betreff: Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost
> configs)
> 
> On Sat, Apr 21, 2018 at 12:45 PM, Graham Leggett <minf...@sharp.fm>
> wrote:
> > On 19 Apr 2018, at 5:55 PM, David Zuelke <dzue...@salesforce.com>
> wrote:
> >
> >> I hate to break this to you, and I do not want to discredit the
> >> amazing work all the contributors here are doing, but httpd 2.4 is of
> >> miserable, miserable quality when it comes to breaks and regressions.
> >>
> >> I maintain the PHP/Apache/Nginx infrastructure at Heroku, and I was
> >> able to use the following httpd releases only in the last ~2.5 years:
> >>
> >> - 2.4.16
> >> - 2.4.18
> >> - 2.4.20
> >> - 2.4.29
> >> -2.4.33
> >>
> >> Mostly because of regressions around mod_proxy(_fcgi), REDIRECT_URL,
> whatever.
> >
> > Did you bring these regressions to our attention? Regressions get fix
> very quickly - there was an 18 month period between 2.4.20 and 2.4.29,
> what stopped it being possible to upgrade in that time?
> 
> I double checked. It was actually 2.4.27, not 2.4.29; 15 months, not 18.
> My bad.
> 
> The issue was the PHP-FPM + mod_proxy_fcgi regression introduced in
> 2.4.21; it got reported pretty quickly but took a while to address.
> 
> It finally got fixed in 2.4.26:
> https://bz.apache.org/bugzilla/show_bug.cgi?id=60576
> 
> But that fix broke SCRIPT_NAME:
> https://bz.apache.org/bugzilla/show_bug.cgi?id=61202
> 
> So 2.4.27 was functional again.
> 
> That means between April 11, 2016, and July 11, 2017, httpd with
> mod_proxy_fcgi and PHP-FPM was broken.
> 
> The original change was made against trunk
> (https://bz.apache.org/bugzilla/show_bug.cgi?id=59618) and then
> backported to 2.4.next.

Which was an unfortunate regression that took long to be fixed correctly,
but as far as I remember it did not involve any new features nor refactoring.
"Just" a bugfix that caused a regression that was hard to fix.

> 
> >
> > (As other people have said, there was no release between 2.4.16 and
> 2.4.18, 2.4.19 was replaced two weeks later, and there were no releases
> for you to have used between v2.4.29 and 2.4.33)
> >
> >> This is not any person's fault. This is the fault of the process. The
> >> process can be repaired: bugfixes only in 2.4.x, do RC cycles for
> >> bugfix releases as well (that alone makes the changelog look a lot
> >> less confusing, which is useful for the project's image, see also the
> >> Nginx marketing/FUD discussion in the other thread), and start
> testing
> >> new features in modules first.
> >
> > Unfortunately this misses a fundamental reality of what the httpd
> project is - we are the foundation under many many other things, and
> when we jump from v2.4.x to v2.6.x, our complete ecosystem above us
> needs to be recompiled.
> 
> Going from 2.4.x to 2.6.0 doesn't mean that 2.4.x would no longer be
> maintained. There could easily be some predictable, defined support
> policy for keeping older versions alive.

Which requires enough manpower to maintain all these branches. Looking at the 
past
15 years we never maintained more than two stable branches at the same time.
This is no argument against 2.6.0 as we currently only maintain 2.4.x, but 
against
having 2.6+n.0 each time we want to add new features and keep maintaining 
2.6+n-x.0
I think experience shows that if we would release 2.6 the activity to backport 
new
features to 2.4 would drop over time.

> 
> The other thing is ABI versus API stability; you could say 2.x.
> versions retain API compatibility, but not ABI compatibility, so while
> modules would have to be rebuilt against newer version series, that
> could in virtually all cases happen without having to touch the
> module's code.

This might work with open source modules, but even here you would lose the 
chance
e.g. on LTS distributions to compile your own Apache with later features being 
on
the same ABI level as the OS delivered Apache and install a OS delivered package
of a module to use it with this instead of needing to compile it on your own.
It does not work at all with closed source modules, because as soon as these
vendors need to recompile you will either have to wait for a longer period of
time or need to upgrade their product as well to a newer version.

> 
> >> It makes such little sense to land h2 support in 2.4.something, as
> >> opposed to having it as an official "brand new, try it out"

Re: Start using RCs (Was: Re: So... when should we do 2.4.34? [WAS: Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)])

[William A Rowe Jr]

On Sun, Apr 22, 2018 at 11:34 AM, Daniel Ruggeri  wrote:
>
> The more I think about it, the more I *really* like a semver-ish
> approach where major represents the ABI that will not be broken, minor
> represents the feature set (for backwards compatibility) and patch
> represents forward-compatible changes/fixes. This may be, in spirit,
> what you are suggesting. I'd further add that no directives are added in
> patch releases, but minor releases would be fair game. How we choose to
> maintain that behind the scenes is a thought exercise. Given that most
> of our users get httpd from distros, I'd be in favor of having only two
> long-lived branches: trunk and release. The distros are already taking
> the point-release they want and curating fixes/changes for it, so they
> will be doing what they do regardless of our direction.

Thought exercise... if it is decided that version 3.5.x added an all around
nice set of features, and we had platform maintainers as committers who
wanted to keep backporting to that tree at this project, well... they could
elect to maintain that version as a long lived branch and collaborate on
it while we went on our merry way through 3.15.0, even 4.1.x as a group.

In other words, wherever we have three committers who will cultivate a
long lived release, they can do that. This doesn't differ from how a few
individuals kept 2.0, and later, 2.2 alive for a significant period of time
after the next major release. When interest cooled, each was retired.

Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)

[David Zuelke]

On Sat, Apr 21, 2018 at 12:45 PM, Graham Leggett  wrote:
> On 19 Apr 2018, at 5:55 PM, David Zuelke  wrote:
>
>> I hate to break this to you, and I do not want to discredit the
>> amazing work all the contributors here are doing, but httpd 2.4 is of
>> miserable, miserable quality when it comes to breaks and regressions.
>>
>> I maintain the PHP/Apache/Nginx infrastructure at Heroku, and I was
>> able to use the following httpd releases only in the last ~2.5 years:
>>
>> - 2.4.16
>> - 2.4.18
>> - 2.4.20
>> - 2.4.29
>> -2.4.33
>>
>> Mostly because of regressions around mod_proxy(_fcgi), REDIRECT_URL, 
>> whatever.
>
> Did you bring these regressions to our attention? Regressions get fix very 
> quickly - there was an 18 month period between 2.4.20 and 2.4.29, what 
> stopped it being possible to upgrade in that time?

I double checked. It was actually 2.4.27, not 2.4.29; 15 months, not 18. My bad.

The issue was the PHP-FPM + mod_proxy_fcgi regression introduced in
2.4.21; it got reported pretty quickly but took a while to address.

It finally got fixed in 2.4.26:
https://bz.apache.org/bugzilla/show_bug.cgi?id=60576

But that fix broke SCRIPT_NAME:
https://bz.apache.org/bugzilla/show_bug.cgi?id=61202

So 2.4.27 was functional again.

That means between April 11, 2016, and July 11, 2017, httpd with
mod_proxy_fcgi and PHP-FPM was broken.

The original change was made against trunk
(https://bz.apache.org/bugzilla/show_bug.cgi?id=59618) and then
backported to 2.4.next.

>
> (As other people have said, there was no release between 2.4.16 and 2.4.18, 
> 2.4.19 was replaced two weeks later, and there were no releases for you to 
> have used between v2.4.29 and 2.4.33)
>
>> This is not any person's fault. This is the fault of the process. The
>> process can be repaired: bugfixes only in 2.4.x, do RC cycles for
>> bugfix releases as well (that alone makes the changelog look a lot
>> less confusing, which is useful for the project's image, see also the
>> Nginx marketing/FUD discussion in the other thread), and start testing
>> new features in modules first.
>
> Unfortunately this misses a fundamental reality of what the httpd project is 
> - we are the foundation under many many other things, and when we jump from 
> v2.4.x to v2.6.x, our complete ecosystem above us needs to be recompiled.

Going from 2.4.x to 2.6.0 doesn't mean that 2.4.x would no longer be
maintained. There could easily be some predictable, defined support
policy for keeping older versions alive.

The other thing is ABI versus API stability; you could say 2.x.
versions retain API compatibility, but not ABI compatibility, so while
modules would have to be rebuilt against newer version series, that
could in virtually all cases happen without having to touch the
module's code.

>> It makes such little sense to land h2 support in 2.4.something, as
>> opposed to having it as an official "brand new, try it out" subproject
>> first, and then bundle it with 2.6.
>
> Not only does it make sense, but it is vital we do so.
>
> We needed to get h2 support into the hands of end users - end users who were 
> not going to recompile their entire web stack, who install software from 
> distros who are not going to upgrade, who were deploying modules from vendors 
> that were not going to recompile.

But that's what I'm saying. Why was h2 not kept as a module (for the
few people that are already deploying HTTP/2 stacks), let it mature
this way, and then put it into everyone's hands as part of 2.6.0,
which could be the big shiny new feature, to give everyone an
incentive to move to that new major version?

> Our average user will deploy whatever comes by default on their operating 
> system, they’re not going to have a dedicated team that deploys a custom 
> stack for their application. It is vital we respect the needs of these groups 
> of users.

That is even more of an argument to move to a more predictable cycle
and have patch releases only fix issues, because it means new features
see the light of day more quickly, so more people who just use what
comes with their OS would benefit from them.

Nobody who uses Apache as part of Debian, Ubuntu, RHEL, whatever, gets
new 2.4.next features. Those distros freeze Apache at whatever is the
latest version when their cutoff date is due, and then only backport
security fixes.

>> Really, I'd suggest taking a close look at the PHP release cycle, with
>> their schedules, their RFC policies, everything. As I said in that
>> other thread, the PHP project was in exactly the same spot a few years
>> ago and they have pulled themselves out of that mess with amazing
>> results.
>
> Specifically what about the php release cycle are you referring to? I was 
> burned badly a number of years ago by php config file formats being changed 
> in point releases, have they improved their stability?

Yes. Nothing like that has happened since PHP 5.4, when the new
release process started and all of 

Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)

[Micha Lenk]

On Fri, Apr 20, 2018 at 08:14:16AM -0400, Jim Jagielski wrote:
> On Apr 20, 2018, at 8:04 AM, Micha Lenk  wrote:
> > [...], I value the ability to distinguish between bugfix-only
> > releases and feature addition releases.
> 
> I understand that, thx. I also understand how a minor bump makes that
> easier. It would make, however, other people's lives and jobs *more*
> difficult, I think, so it's all about balance. I can see how
> distinguishing the difference is also nice, but what "value" do you
> derive from it? I am genuinely curious. Thx!

To be honest, our commercial interest in bugfixes is simply higher than
getting new features. So, I expect integrating a bugfix-only release to
be much less effort (in terms of porting our own modules, patches,
additional testing scrutiny) than a release that re-works internal core
functionality like the request handling for the sake of adding a new
feature like the entirely new support for h2.

But I am equally genuinely curious what would make "other people's lives
and jobs *more* difficult". What exactly do you refer to here?

> This is a "hack", of course, but what if CHANGES specifically called
> out new features like we do w/ SECURITY items?

Not being a native speaker I am not sure I understand your question
correctly. Can you please elaborate that question a bit?

Regards,
Micha

Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)

[Alain Toussaint]

Le mercredi 18 avril 2018 à 19:31 -0500, Daniel Ruggeri a écrit :
> On 4/18/2018 1:34 PM, Alain Toussaint wrote:
> > > As an aside - httpd has a —enable-layout option in configure that defines 
> > > where things should
> > > go.
> > > If you patch the following file how you want it and submit it to us, we 
> > > can formally support
> > > LFS
> > > out the box and you can remove the need for your patch:
> > > 
> > > https://svn.apache.org/repos/asf/httpd/sandbox/replacelimit/config.layout
> > > 
> > > Regards,
> > > Graham
> > > —
> > > 
> > 
> > Great idea which I'll submit to the power that be.
> > 
> > Alain
> 
> Minor correction to the URL for latest and greatest:
> https://svn.apache.org/repos/asf/httpd/httpd/trunk/config.layout
> 
> As we love to say, "patches welcome!"
> 
> Feel free to just submit your diff here (since dev@ IS the power that be)
> 

I've been tasked with the patch modification at BLFS. I'll handle it tomorrow 
morning and submit it.

Alain

Re: Start using RCs (Was: Re: So... when should we do 2.4.34? [WAS: Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)])

[Greg Stein]

On Sun, Apr 22, 2018 at 11:34 AM, Daniel Ruggeri 
wrote:

> Yeah - I think my main concern is really just around the backporting
> process with STATUS and how that will have issues scaling across many
> branches. Further, as each branch deviates, it becomes more of a
> cognitive exercise for developers to figure out if the fix in branch XYZ
> is applicable and the same in ABC.
>
> The more I think about it, the more I *really* like a semver-ish
> approach where major represents the ABI that will not be broken, minor
> represents the feature set (for backwards compatibility) and patch
> represents forward-compatible changes/fixes.


Apache Subversion takes the patch number much more literally. They are
merely fixes to behavior defined in the minor release. The API/ABI is never
touched during a patch release. You could build against 1.7.13, then
install 1.7.0 and your code works against it. Then upgrade svn to 1.7.20
and it still works. Downgrade to 1.7.5 ... you get the picture.

I see several issues with the recent threads of discussion:

* How releases are numbered, and what guarantees are made relative to those
numbers
* What kinds of gating of features/fixes is defined by the process?
* How are branches created/maintained, relative to the above

Classic httpd numbering can certainly be made to work (empirically: it
has). semver is well-documented, understood, and downstream users would not
need to understand our quirks. It does kind of impose decisions on gating
of features, though (Q2 above).

It is unclear what problem is being solved. It seems like the unpredictable
feature set of 2.4.x. And/or when to stop loading features into that, and
start a 2.6.x series. (or 3.0?)

Cheers,
-g

ps. my vote is for semver and strict controls on patch releases. chew
through release numbers. they are cheap. downstream will pick one release
and run with that. it doesn't matter to them if we have a new minor every
six months (which means new features, which distros won't pick those
features from patch releases anyways; may as well move them to a minor)

Re: Start using RCs (Was: Re: So... when should we do 2.4.34? [WAS: Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)])

[Eric Covener]

On Sun, Apr 22, 2018 at 2:03 PM, Daniel Ruggeri  wrote:
> Unhelpful for whom? If we ship the latest, secure config from the single 
> release branch, we wouldn't be encumbered by having to use tricks for fixes.

I think we're getting off into the weeds a bit here.  My belief is
that most users extend the configuration, and different users want
different behaviors even if they use the same distribution.  Most
users  at any given time want no changes at all beyond the required
security fixes.

I don't feel like "directives" are even a secondary source of any of
the recent regressions. In reality, gating changes behind directives
would have likely avoided a good deal of the regressions if we had a
different tolerance for such a thing or the impacts could have been
anticipated.

That's why I don't see any benefit in prohibiting new directives in a
reworked service stream being managed as more stable, even without
weighing any of the other tradeoffs. I see only risk in that fixes can
only be delivered when they are safe for 100% of users 100% of the
time.

> In the same vein of thought, if it is disruptive to a config, that signals a 
> minor bump. Patch changes must be forward compatible.

This doesn't really differ from the status quo.

Re: Start using RCs (Was: Re: So... when should we do 2.4.34? [WAS: Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)])

[Graham Leggett]

On 19 Apr 2018, at 5:11 PM, Jim Jagielski  wrote:

> With all this in mind, should we try to set things up so that the
> next release cycle uses the concept of RCs?
> 
> If so, and if people like, I can come up with a baseline
> proposal on the process for us to debate and come to
> some consensus on.

What specific problem does this aim to solve?

Regards,
Graham
—



smime.p7s
Description: S/MIME cryptographic signature

Re: Start using RCs (Was: Re: So... when should we do 2.4.34? [WAS: Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)])

[Daniel Ruggeri]

Unhelpful for whom? If we ship the latest, secure config from the single 
release branch, we wouldn't be encumbered by having to use tricks for fixes. 
Most end users who get their custom build from the distros would get the same 
custom directive-level fixes applicable to them (ie:  they know what's in the 
build so don't have to do such gymnastics).

In the same vein of thought, if it is disruptive to a config, that signals a 
minor bump. Patch changes must be forward compatible.
-- 
Daniel Ruggeri

On April 22, 2018 12:32:24 PM CDT, Eric Covener  wrote:
>On Sun, Apr 22, 2018 at 1:31 PM, Eric Covener 
>wrote:
>>> I'd further add that no directives are added in
>>> patch releases, but minor releases would be fair game.
>>
>> I don't think that kind of rule would be helpful, it just pushes
>> configuration into defines, per-request environment variables, etc
>> which are worse (because unlike a directive, you can't tell if
>they're
>> present)
>
>Of fixes which may be disruptive, that is.

Re: Start using RCs (Was: Re: So... when should we do 2.4.34? [WAS: Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)])

[Eric Covener]

On Sun, Apr 22, 2018 at 1:31 PM, Eric Covener  wrote:
>> I'd further add that no directives are added in
>> patch releases, but minor releases would be fair game.
>
> I don't think that kind of rule would be helpful, it just pushes
> configuration into defines, per-request environment variables, etc
> which are worse (because unlike a directive, you can't tell if they're
> present)

Of fixes which may be disruptive, that is.

Re: Start using RCs (Was: Re: So... when should we do 2.4.34? [WAS: Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)])

[Eric Covener]

> I'd further add that no directives are added in
> patch releases, but minor releases would be fair game.

I don't think that kind of rule would be helpful, it just pushes
configuration into defines, per-request environment variables, etc
which are worse (because unlike a directive, you can't tell if they're
present)

Re: Start using RCs (Was: Re: So... when should we do 2.4.34? [WAS: Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)])

[Daniel Ruggeri]

Yeah - I think my main concern is really just around the backporting
process with STATUS and how that will have issues scaling across many
branches. Further, as each branch deviates, it becomes more of a
cognitive exercise for developers to figure out if the fix in branch XYZ
is applicable and the same in ABC.

The more I think about it, the more I *really* like a semver-ish
approach where major represents the ABI that will not be broken, minor
represents the feature set (for backwards compatibility) and patch
represents forward-compatible changes/fixes. This may be, in spirit,
what you are suggesting. I'd further add that no directives are added in
patch releases, but minor releases would be fair game. How we choose to
maintain that behind the scenes is a thought exercise. Given that most
of our users get httpd from distros, I'd be in favor of having only two
long-lived branches: trunk and release. The distros are already taking
the point-release they want and curating fixes/changes for it, so they
will be doing what they do regardless of our direction. For
administrators building httpd themselves, they have control of when to
pull the trigger on a build and what they build, so I don't think they
would be harmed by such a change. Hopefully this isn't considered a
caustic statement (it is not intended to be), but I feel that this would
bring our versioning practices more in line with what most OSS projects
are doing these days and also avoid some of the issues managing several
branches. (I really am digging this idea :-) I'm glad folks have
proposed it...)

-- 
Daniel Ruggeri

On 4/20/2018 12:08 AM, William A Rowe Jr wrote:
> Let me counter with this... Rather than break the API every m.n
> release, what if we roll on to 2.6 with no ABI breakage, or resolve
> with impumity everything wrong in 3.0 with a firm commitment not to
> break it again till 4.0?
>
> This might be the root problem of our trying to !is versioning semantics.
>
>
> On Thu, Apr 19, 2018, 19:39 Daniel Ruggeri  > wrote:
>
> I'm not sure where in the conversation to add this, but I do want to
> point out a mechanical concern.
>
>
> If we end up with API and feature freeze on branch 2.4, then we'd
> expect
> to roll 2.6. Soon enough, we'll hit a situation where 2.6 (as a
> release
> branch) can't get a feature or the next great thing because of a
> required API incompatible change. We would then kick out 2.8, 2.10 and
> so on and so forth. This seems like it would satisfy both the
> keep-the-stuff-stable as well as the give-users-new-stuff "sides".
>
>
> Mechanically, this can become tiresome for a volunteer as we work
> through the STATUS ceremony for each of the branches. I remember that
> being less than enjoyable when 2.0, 2.2 and 2.4 were all release
> branches. I'm fearful of a future where we may have five branches
> and a
> bugfix applicable to all (or worse... a security fix that would
> dictate
> all should be released/disclosed at the same time).
>
> -- 
> Daniel Ruggeri
>
> On 4/19/2018 7:17 PM, William A Rowe Jr wrote:
> > I don't disagree with RC's entirely, or the mechanism you
> suggest, but
> > that isn't what I read as proposed.
> >
> > Our issue is that every httpd must be distinguished, we won't ship
> > four tarballs all claiming 2.4.34 (GA). Which one is the person
> > complaining about? Are we back to SHA signature of the source
> tarball
> > (that isn't even assuredly what they built the binary from?)
> >
> > We can decorate the rc in the versioning, but that isn't part of
> 2.4.x
> > API, unless we swap out the "-dev" string tag for an "-rc#" value.
> >
> > It is not reasonable to lock a branch for an indeterminate length of
> > time. Branching the RC into what becomes the final release, and
> making
> > it painful to backport first to 2.4.x and finally 2.4.n-rc branch
> > might help discourage disruptive backports, but there is no
> suggestion
> > yet of what is acceptable, either on such a frozen main branch or rc
> > branch.
> >
> > In fact our policy reflects that two competing release candidates is
> > an entire valid and even expected situation, and any process that
> > further blocks this should be firmly rebuffed.
> >
> > As it reads so far the proposal is the way we do things, now
> > conserving subversion numbers as precious, if only to reenforce a
> > stake in the ground that version major and minor numbers are
> similarly
> > precious, (which they are not.)
> >
> > With the addition of freezing changes on 2.4.x branch for a time we
> > succeed in impeading progress towards version .next, while not
> > otherwise changing the status quo.
> >
> > What you suggest is yet another thing, based on forking the RC
> release
>  

Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)

[Graham Leggett]

On 19 Apr 2018, at 10:35 PM, David Zuelke  wrote:

> Of course, but that's exactly my point. It was introduced not in
> 2.4.0, but in 2.4.17. Five "H2…" config directives are available in
> 2.4.18+ only, one in 2.4.19+, and three in 2.4.24+.

H2 support was marked as “experimental” in the versions you are listing, and so 
changes to these directives were entirely expected as per our process.

As per our changelog, H2 support was marked as fully production ready and 
therefore subject to our normal versioning rules as of v2.4.26:

http://www-eu.apache.org/dist//httpd/CHANGES_2.4

Our process allows the introduction of new features for the reasons already 
explained previously.

Regards,
Graham
—



smime.p7s
Description: S/MIME cryptographic signature

Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)

[Graham Leggett]

On 19 Apr 2018, at 5:55 PM, David Zuelke  wrote:

> I hate to break this to you, and I do not want to discredit the
> amazing work all the contributors here are doing, but httpd 2.4 is of
> miserable, miserable quality when it comes to breaks and regressions.
> 
> I maintain the PHP/Apache/Nginx infrastructure at Heroku, and I was
> able to use the following httpd releases only in the last ~2.5 years:
> 
> - 2.4.16
> - 2.4.18
> - 2.4.20
> - 2.4.29
> -2.4.33
> 
> Mostly because of regressions around mod_proxy(_fcgi), REDIRECT_URL, whatever.

Did you bring these regressions to our attention? Regressions get fix very 
quickly - there was an 18 month period between 2.4.20 and 2.4.29, what stopped 
it being possible to upgrade in that time?

(As other people have said, there was no release between 2.4.16 and 2.4.18, 
2.4.19 was replaced two weeks later, and there were no releases for you to have 
used between v2.4.29 and 2.4.33)

> This is not any person's fault. This is the fault of the process. The
> process can be repaired: bugfixes only in 2.4.x, do RC cycles for
> bugfix releases as well (that alone makes the changelog look a lot
> less confusing, which is useful for the project's image, see also the
> Nginx marketing/FUD discussion in the other thread), and start testing
> new features in modules first.

Unfortunately this misses a fundamental reality of what the httpd project is - 
we are the foundation under many many other things, and when we jump from 
v2.4.x to v2.6.x, our complete ecosystem above us needs to be recompiled.

This cannot be ignored, understated or taken lightly.

> It makes such little sense to land h2 support in 2.4.something, as
> opposed to having it as an official "brand new, try it out" subproject
> first, and then bundle it with 2.6.

Not only does it make sense, but it is vital we do so.

We needed to get h2 support into the hands of end users - end users who were 
not going to recompile their entire web stack, who install software from 
distros who are not going to upgrade, who were deploying modules from vendors 
that were not going to recompile.

Our average user will deploy whatever comes by default on their operating 
system, they’re not going to have a dedicated team that deploys a custom stack 
for their application. It is vital we respect the needs of these groups of 
users.

> Speaking of which, I'd also suggest dropping this odd/even number
> meaning experimental/stable versioning scheme, since it only
> aggravates the problem: never-ending experiments that go stale, maybe
> even get half backported, and meanwhile are subconsciously perceived
> as one more hurdle towards a next bigger release.

I don’t see us having any of the problems you describe above to the extent 
where it would be a real problem. Part of the process of creating an odd 
numbered branch is to determine what from trunk gets carried over and what is 
not, this is all catered for in the process.

> Really, I'd suggest taking a close look at the PHP release cycle, with
> their schedules, their RFC policies, everything. As I said in that
> other thread, the PHP project was in exactly the same spot a few years
> ago and they have pulled themselves out of that mess with amazing
> results.

Specifically what about the php release cycle are you referring to? I was 
burned badly a number of years ago by php config file formats being changed in 
point releases, have they improved their stability?

> I am also happy to make introductions to release managers and
> maintainers there. Heck I am betting some of them would happily serve
> as tutors for the httpd project ;) I'm certainly willing to help too.
> But IMO you need a clean cut and shake up the entire process, not just
> a little, because otherwise you won't get rid of some of the old
> habits that have been plaguing the project.

There is a fundamental tension between two groups of people - people who want 
stability, and people who want features.

I believe the httpd project has been very successful at trading off against 
these two, and other projects have a lot to learn from us.

Regards,
Graham
—



smime.p7s
Description: S/MIME cryptographic signature

Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)

[William A Rowe Jr]

On Fri, Apr 20, 2018, 10:37 Paul Querna  wrote:
>
> I believe having more minor releases and less major backports to patch
> releases is a good thing.
>
> I believe we gave the even/odd, 2.1/2.3 "unstable", thing a long run.
> About 15 years of it.
>
> Since then the wider open source world has gone to a more canonical
> semver.  I think we should generally align with that.
> 
>
> As an outcome, that would mean more major & minor releases, and less
> patch releases.  I think that if we break an existing ABI, we need to
> bump the major.  I think other projects are successfully doing this,
> and it has little effect on how distributions are packaging their
> projects.  Distros pick a point in time, whatever the current
> major.minor.patch is.

According to my math and history here, which begins with binary stability
(and not much feature changes in the period) at 1.3.14, and continuing
throughout all of 2.0/2.2/2.4...

Those would have been counted, using any semver scheme, as 2.0+++, 3.0+++,
4.0+++. Over the span of 18 years.

That would put us at approaching 5.0.0, with discussion of refactoring how
our URI's can successfully handle %CH entities correctly and finally close
a group of (now mitigated but still broken) security issues. Mop up some
other crufty bits in the process. Perhaps achieve 99.9% source
compatibility with judicious use of compatibility macros.

2.0+ major revisions (ABI stable-ish) persisted over 6 years each. That
isn't terribly shabby. Future majors would be even less frequent, as the
framework proves durable.

Pretend we are at 4.x, what would be our minor? I count only 21 releases in
2.4.x and 3 of those were immediate reactions to busted releases. That
corresponds to releasing 4.17.1 in the span of 6 years. (I'm not suggesting
renumbering, obviously... only trying to frame our project in semver terms
to test if this would be helpful or harmful.)

About 3 minor releases per year, and even that was perhaps too infrequent.
The subversion bug-fix releases were certainly too infrequent, we shouldn't
have waited so long on the fixes, but new development, and yes, refactoring
also slow us down a lot.

Thanks for the observations!

Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)

[Paul Querna]

I believe having more minor releases and less major backports to patch
releases is a good thing.

I believe we gave the even/odd, 2.1/2.3 "unstable", thing a long run.
About 15 years of it.

Since then the wider open source world has gone to a more canonical
semver.  I think we should generally align with that.


As an outcome, that would mean more major & minor releases, and less
patch releases.  I think that if we break an existing ABI, we need to
bump the major.  I think other projects are successfully doing this,
and it has little effect on how distributions are packaging their
projects.  Distros pick a point in time, whatever the current
major.minor.patch is.


On Fri, Apr 20, 2018 at 5:04 AM, Micha Lenk  wrote:
> Hi all,
>
> On 04/20/2018 01:34 PM, Jim Jagielski wrote:
>>
>> But why does it matter that h2 was added in 2.4.x instead of
>> a 2.6.0?
>
>
> Because it sets a bad precedence (or even continues to do so)?
>
>> Every new feature must bump the minor? Even if
>> there is no corresponding ABI issue?
>
>
> Why not?
>
> In my role as Debian Developer maintaining the Debian packages of other OSS
> projects, and also in my role of maintaining a commercial reverse proxy
> product based on Apache httpd during my day job, I value the ability to
> distinguish between bugfix-only releases and feature addition releases.
>
> This does not mean that a minor bump needs to happen at almost every
> release. But not bumping the minor for years (which seems to be the current
> pattern of the httpd project) is just worse, because it increases the
> incentive to squeeze features like h2 into releases that are meant (or
> perceived) as bugfix-only releases.
>
> Regards,
> Micha

Re: Start using RCs (Was: Re: So... when should we do 2.4.34? [WAS: Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)])

[Jim Jagielski]

> On Apr 20, 2018, at 8:28 AM, Micha Lenk  wrote:
> 
> Hi Jim,
> 
> On 04/20/2018 01:46 PM, Jim Jagielski wrote:
>> Where numbers and versioning DOES matter is how it affects
>> distributors and vendors of httpd and the entire module eco-system.
> No, it doesn't. There are way too many variants of versioning schemes out 
> there in use by so many OSS projects that the distributors and vendors need 
> to be prepared to encounter any variant you can imagine.
> 

We have a history, as well as a published "agreement" on what minor version 
numbering means. Our module eco-system is huge and we need to factor/consider 
not only the big players, but also the solitary developers who have modules. It 
is a long, long history and if/when we need to reconsider versioning, the 
impact will not be insignificant.

> What matters is quality (here I do agree with you). A versioning scheme can 
> help to establish certain rules of what to do and more importantly what to 
> *not* do on a given version pattern or branch. And with the current rate of 
> successful releases, apparently the current rules either were not enforced 
> well enough or otherwise were not good enough and thus need to be changed. A 
> new versioning scheme then could help to establish new, hopefully better 
> rules.

Or just change the goalposts... if, as you say, our current version-scheme 
rules don't work as far as limiting/controlling what we can do (which I don't 
agree with, btw), then simply replacing it with another version-scheme likely 
won't do anything as well.

Again, I don't think numbers or names are the issue at hand, at core, but 
rather, maybe, overconfidence in how we have been doing release QA and testing. 
For example, and I am certainly guilty of this, the original concept was that 
"new stuff" was put in trunk, and allowed to mellow and age, for a "bit" at 
least, to get a feel for how well it works. After "awhile", when things looked 
like it was "ready for prime time", it would be proposed for backport. Today, 
right after something is committed to trunk there is almost invariably, in the 
next commit, a backport proposal for it to be in 2.4. Now for bug fixes, etc, 
that likely makes some level of sense, but lately, it seems like committing to 
trunk is just a tic-mark.

Again, it is all in balance, and likely we've just not achieved that lately due 
to the extreme complexities of the eco-system around, internal, external and 
dependent upon us.

Re: Start using RCs (Was: Re: So... when should we do 2.4.34? [WAS: Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)])

[Micha Lenk]

Hi Jim,

On 04/20/2018 01:46 PM, Jim Jagielski wrote:

Where numbers and versioning DOES matter is how it affects
distributors and vendors of httpd and the entire module eco-system.
No, it doesn't. There are way too many variants of versioning schemes 
out there in use by so many OSS projects that the distributors and 
vendors need to be prepared to encounter any variant you can imagine.


What matters is quality (here I do agree with you). A versioning scheme 
can help to establish certain rules of what to do and more importantly 
what to *not* do on a given version pattern or branch. And with the 
current rate of successful releases, apparently the current rules either 
were not enforced well enough or otherwise were not good enough and thus 
need to be changed. A new versioning scheme then could help to establish 
new, hopefully better rules.


Just my 2¢.

Regards,
Micha

Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)

[Jim Jagielski]

> On Apr 20, 2018, at 8:04 AM, Micha Lenk  wrote:
> 
> In my role as Debian Developer maintaining the Debian packages of other OSS 
> projects, and also in my role of maintaining a commercial reverse proxy 
> product based on Apache httpd during my day job, I value the ability to 
> distinguish between bugfix-only releases and feature addition releases.
> 

I understand that, thx. I also understand how a minor bump makes that easier. 
It would make, however, other people's lives and jobs *more* difficult, I 
think, so it's all about balance. I can see how distinguishing the difference 
is also nice, but what "value" do you derive from it? I am genuinely curious. 
Thx!

This is a "hack", of course, but what if CHANGES specifically called out new 
features like we do w/ SECURITY items?

Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)

[Micha Lenk]

Hi all,

On 04/20/2018 01:34 PM, Jim Jagielski wrote:

But why does it matter that h2 was added in 2.4.x instead of
a 2.6.0?


Because it sets a bad precedence (or even continues to do so)?


Every new feature must bump the minor? Even if
there is no corresponding ABI issue?


Why not?

In my role as Debian Developer maintaining the Debian packages of other 
OSS projects, and also in my role of maintaining a commercial reverse 
proxy product based on Apache httpd during my day job, I value the 
ability to distinguish between bugfix-only releases and feature addition 
releases.


This does not mean that a minor bump needs to happen at almost every 
release. But not bumping the minor for years (which seems to be the 
current pattern of the httpd project) is just worse, because it 
increases the incentive to squeeze features like h2 into releases that 
are meant (or perceived) as bugfix-only releases.


Regards,
Micha

Re: So... when should we do 2.4.34? [WAS: Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)]

[Rainer Jung]

Am 20.04.2018 um 11:39 schrieb Eric Covener:

On Fri, Apr 20, 2018 at 3:15 AM, Rainer Jung  wrote:

Do we need a quick APR 1.6.4 to pick up r1819938? From CHANGES:

*) poll, port: re-add the wakeup pipe to the pollset after it triggered.
Not doing this occasionally lead to httpd event MPM processes hanging
during process shutdown.  PR 61786.  [Yann Ylavic]



Never hurts, but it is just Solaris isn't it?


I think so for the CHANGES topic (ports impl of poll), although there 
were more changes in the backport commit for which I'm not sure about 
how important they are and when they apply.


Regards,

Rainer

Re: Start using RCs (Was: Re: So... when should we do 2.4.34? [WAS: Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)])

[Jim Jagielski]

> On Apr 20, 2018, at 1:08 AM, William A Rowe Jr  wrote:
> 
> Let me counter with this... Rather than break the API every m.n release, what 
> if we roll on to 2.6 with no ABI breakage, or resolve with impumity 
> everything wrong in 3.0 with a firm commitment not to break it again till 4.0?

Doesn't that break our understanding/"contract" with all external module 
developers and ISV? Up until now, minor has always indicated ABI changes which 
means, real world, you need to provide a different module build for each minor 
version. Now we would be saying "Nope, not in this case"

That seems, to me, worse than what we have now. Plus, it does nothing to 
satisfy the concerns about "quality" of what is released in any way.

Again, IMO, it's not about numbers, or what we call it, for US. It about QA, 
testing and being more sensitive to regression and wholesale refactoring on 
stuff that is going to be released. Where numbers and versioning DOES matter is 
how it affects distributors and vendors of httpd and the entire module 
eco-system.

Just my 2c

Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)

[Jim Jagielski]

> On Apr 19, 2018, at 4:35 PM, David Zuelke  wrote:
> 
> 
> Of course, but that's exactly my point. It was introduced not in
> 2.4.0, but in 2.4.17. Five "H2…" config directives are available in
> 2.4.18+ only, one in 2.4.19+, and three in 2.4.24+.
> 

But why does it matter that h2 was added in 2.4.x instead of
a 2.6.0? Every new feature must bump the minor? Even if
there is no corresponding ABI issue?

You wrote:
  It makes such little sense to land h2 support in 2.4.something, as
  opposed to having it as an official "brand new, try it out" subproject
  first, and then bundle it with 2.6.

h2 was a '"brand new, try it out" subproject', external
to httpd. It was brought in via a generous donation of
code and because there was a desire and "need" for it to
be an official part of the distro.

In general, even though PHP says that all "New feature"
must be in a Minor bump, this has not ever been the
case for httpd. Otherwise we'd be up to version 2.223.x
by now, after having left version 1.6542.x :)

And finally, it's not quite apples/apples comparing
language version numbering to *server software* versioning,
especially when there is a large, external module eco-system
for the server software that relies on the minor number
being "only" ABI related.

Re: So... when should we do 2.4.34? [WAS: Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)]

[Eric Covener]

On Fri, Apr 20, 2018 at 3:15 AM, Rainer Jung  wrote:
> Do we need a quick APR 1.6.4 to pick up r1819938? From CHANGES:
>
> *) poll, port: re-add the wakeup pipe to the pollset after it triggered.
>Not doing this occasionally lead to httpd event MPM processes hanging
>during process shutdown.  PR 61786.  [Yann Ylavic]


Never hurts, but it is just Solaris isn't it?

Re: So... when should we do 2.4.34? [WAS: Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)]

[William A Rowe Jr]

On Fri, Apr 20, 2018 at 2:36 AM, Rainer Jung  wrote:
>
>> The necessity/evaluation of the dependency belongs
>> here... we once could compile httpd 2.4 against APR 1.4 family and had
>> agreement that this would continue through the lifecycle, but I seriously
>> doubt that is still true today.
>
> If that is a concern, it would be better to check and if needed start an
> independent discussion thread for that.

The logic I pointed Jeff at has four orientations, stable, candidate, snapshot
and bleed (for future revs, e.g. trunk/ across the board.)

What oss-httpd-build doesn't do (at least not yet) are all the permutations
of very old components, e.g. APR gen 1.4.x + httpd. I don't have enough
cycles to tackle it all, but as the Makefile.checkout phase is enhanced,
that's one really interesting aspect to consider (1.4.x gen, 1.5.x gen, the
current 1.6.x gen and eventually 1.7.x APR.)

Re: Start using RCs (Was: Re: So... when should we do 2.4.34? [WAS: Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)])

[Mark Blackman]

> On 20 Apr 2018, at 01:39, Daniel Ruggeri  wrote:
> 
> I'm not sure where in the conversation to add this, but I do want to
> point out a mechanical concern.
> 
> 
> If we end up with API and feature freeze on branch 2.4, then we'd expect
> to roll 2.6. Soon enough, we'll hit a situation where 2.6 (as a release
> branch) can't get a feature or the next great thing because of a
> required API incompatible change. We would then kick out 2.8, 2.10 and
> so on and so forth. This seems like it would satisfy both the
> keep-the-stuff-stable as well as the give-users-new-stuff "sides".
> 
> 
> Mechanically, this can become tiresome for a volunteer as we work
> through the STATUS ceremony for each of the branches. I remember that
> being less than enjoyable when 2.0, 2.2 and 2.4 were all release
> branches. I'm fearful of a future where we may have five branches and a
> bugfix applicable to all (or worse... a security fix that would dictate
> all should be released/disclosed at the same time).

Presumably most/all of that toil can be automated away as you have started 
doing, so that human intervention is only required where human judgement is 
actually required?  Does httpd have some massive CD/CI pipeline in the 
background I don’t see or could it do with one? 

- Mark

Re: Start using RCs (Was: Re: So... when should we do 2.4.34? [WAS: Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)])

[Ruediger Pluem]

On 04/19/2018 09:19 PM, Rainer Jung wrote:
> Am 19.04.2018 um 17:37 schrieb Jim Jagielski:
>>
>>
>>> On Apr 19, 2018, at 11:26 AM, William A Rowe Jr  wrote:
>>>
>>> On Thu, Apr 19, 2018 at 10:11 AM, Jim Jagielski  wrote:
 With all this in mind, should we try to set things up so that the
 next release cycle uses the concept of RCs?

 If so, and if people like, I can come up with a baseline
 proposal on the process for us to debate and come to
 some consensus on.
>>>
>>> Would you define an RC? What changes are allowable in that branch?
>>
>>
>> The idea is that right now we take an existing state in SVN
>> and tag it as, for example, 2.4.121. We then vote on that tag
>> and the artifacts released from that tag. No work is done on
>> the 2.4 branch while testing is done so that we ensure that
>> the SVN rev on branch == the one for the tag.
>>
>> Instead, we tag at 2.4.121-RC1. We do the exact same. If the
>> vote does not pass, we continue on the 2.4 branch, fix the
>> issues, and then tag a 2.4.121-RC2. Rinse and repeat.
>>
>> If the vote does pass we tag the branch, which is still == RC#
>> at this stage, as 2.4.121 (ap_release.h mods included). We
>> still even at this stage test and vote on the release as we have
>> for decades. If it passes, we release. If not, for some reason,
>> we have burned the 2.4.121 release, bump to 2.4.122 and GOTO
>> the above.
>>
>> This is the overall idea... more flesh to be added.
> 
> IMHO we should aim at keeping the RC phase as stable as possible and focuse 
> on the code from which we tagged RC1 and
> which we actually want to release. So after RC1 there should be only fixes 
> for regressions, no other fixes or backports.
> 
> Other projects for example branch at the start of the release process (using 
> your version example 2.4.121 here):
> 
> - branch 2.4.x as a 2.4.121 branch
> - tag 2.4.121-RC1 on that branch
> - allow for a week or two (?) of public testing
> - fix incoming regressions
>   - patches go via trunk to 2.4.x to 2.4.121 branch
> - cut new RCs if previous RC needed fixes
> - create final release tag from last RC plus some script driven version 
> adjustments
> - do final release vote

+1. Sounds reasonable, gives more time for testing avoids freezes on the 2.4.x 
branch while keeping the RC stable. Let's
try this and see the results.

Regards

Rüdiger

Re: So... when should we do 2.4.34? [WAS: Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)]

[Rainer Jung]

Am 20.04.2018 um 09:22 schrieb William A Rowe Jr:

On Fri, Apr 20, 2018 at 2:15 AM, Rainer Jung  wrote:

Do we need a quick APR 1.6.4 to pick up r1819938? From CHANGES:

*) poll, port: re-add the wakeup pipe to the pollset after it triggered.
Not doing this occasionally lead to httpd event MPM processes hanging
during process shutdown.  PR 61786.  [Yann Ylavic]


That's an interesting question, would you propose a release to the APR
developer's list?


I will, but I wanted to first check this from the point of view of hte 
httpd project as a consumer of APR/APU and as a topic that might 
influence release timing.



The necessity/evaluation of the dependency belongs
here... we once could compile httpd 2.4 against APR 1.4 family and had
agreement that this would continue through the lifecycle, but I seriously
doubt that is still true today.


If that is a concern, it would be better to check and if needed start an 
independent discussion thread for that.


Regards,

Rainer

Re: So... when should we do 2.4.34? [WAS: Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)]

[William A Rowe Jr]

On Fri, Apr 20, 2018 at 2:15 AM, Rainer Jung  wrote:
> Do we need a quick APR 1.6.4 to pick up r1819938? From CHANGES:
>
> *) poll, port: re-add the wakeup pipe to the pollset after it triggered.
>Not doing this occasionally lead to httpd event MPM processes hanging
>during process shutdown.  PR 61786.  [Yann Ylavic]

That's an interesting question, would you propose a release to the APR
developer's list? The necessity/evaluation of the dependency belongs
here... we once could compile httpd 2.4 against APR 1.4 family and had
agreement that this would continue through the lifecycle, but I seriously
doubt that is still true today.

Re: So... when should we do 2.4.34? [WAS: Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)]

[Rainer Jung]

Do we need a quick APR 1.6.4 to pick up r1819938? From CHANGES:

*) poll, port: re-add the wakeup pipe to the pollset after it triggered.
   Not doing this occasionally lead to httpd event MPM processes hanging
   during process shutdown.  PR 61786.  [Yann Ylavic]

From the commit log:



Merge r1819857, r1819858, r1819860, r1819861, r1819934, r1819935 from trunk:

testpoll: check that the wakeup pipe is still in the pollset after 
returning from poll(), e.g. APR_POLLSET_PORT should re-arm it automatically.



poll, port: re-add the wakeup pipe to the pollset after it triggered.

Just like user fds (file, socket), otherwise it's one shot only (PR-61786).

Corresponding test committed in r1819857.


poll, port: no need to release and re-acquire the lock in between 
walking the pollset and handling the dead ring, all is 
simple/fast/nonblocking ops.


Also, set types of "i" and "j" respectively to the ones of nget and *num.


poll, port: follow up to r1819860.

This test is redundant now, axe it (no functional change).


poll, kqueue: save a pollfd (mem)copy per returned event.


poll, epoll: pollset's pfd is not modified on poll(), mark it const.




At least on Solaris this looks useful. Or do we think that change is to 
big to put it into the deps tarball?



And for APR-Util there is r1825312. From CHANGES:

*) apr_dbm_gdbm: Fix handling of error codes. This makes gdbm 1.14 work.
   apr_dbm_gdbm will now also return error codes starting with
   APR_OS_START_USEERR, as apr_dbm_berkleydb does, instead of always
   returning APR_EGENERAL. [Stefan Fritsch]

and from the commit log:



Fix error handling in gdbm

Only check for gdbm_errno if the return value of the called gdbm_*
function says so. This fixes apr-util with gdbm 1.14, which does not
seem to always reset gdbm_errno.

Also make the gdbm driver return error codes starting with
APR_OS_START_USEERR instead of always returning APR_EGENERAL. This is
what the berkleydb driver already does.

Also ensure that dsize is 0 if dptr == NULL.

(backport of r1825311 in apr trunk)



The risk of breaking things is formally smaller, because builders can 
always switch back to the old dependency tarball without the need for a 
new httpd release, although that situation would produce a bit of confusion.


Regards,

Rainer

Re: Start using RCs (Was: Re: So... when should we do 2.4.34? [WAS: Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)])

[William A Rowe Jr]

Let me counter with this... Rather than break the API every m.n release,
what if we roll on to 2.6 with no ABI breakage, or resolve with impumity
everything wrong in 3.0 with a firm commitment not to break it again till
4.0?

This might be the root problem of our trying to !is versioning semantics.


On Thu, Apr 19, 2018, 19:39 Daniel Ruggeri  wrote:

> I'm not sure where in the conversation to add this, but I do want to
> point out a mechanical concern.
>
>
> If we end up with API and feature freeze on branch 2.4, then we'd expect
> to roll 2.6. Soon enough, we'll hit a situation where 2.6 (as a release
> branch) can't get a feature or the next great thing because of a
> required API incompatible change. We would then kick out 2.8, 2.10 and
> so on and so forth. This seems like it would satisfy both the
> keep-the-stuff-stable as well as the give-users-new-stuff "sides".
>
>
> Mechanically, this can become tiresome for a volunteer as we work
> through the STATUS ceremony for each of the branches. I remember that
> being less than enjoyable when 2.0, 2.2 and 2.4 were all release
> branches. I'm fearful of a future where we may have five branches and a
> bugfix applicable to all (or worse... a security fix that would dictate
> all should be released/disclosed at the same time).
>
> --
> Daniel Ruggeri
>
> On 4/19/2018 7:17 PM, William A Rowe Jr wrote:
> > I don't disagree with RC's entirely, or the mechanism you suggest, but
> > that isn't what I read as proposed.
> >
> > Our issue is that every httpd must be distinguished, we won't ship
> > four tarballs all claiming 2.4.34 (GA). Which one is the person
> > complaining about? Are we back to SHA signature of the source tarball
> > (that isn't even assuredly what they built the binary from?)
> >
> > We can decorate the rc in the versioning, but that isn't part of 2.4.x
> > API, unless we swap out the "-dev" string tag for an "-rc#" value.
> >
> > It is not reasonable to lock a branch for an indeterminate length of
> > time. Branching the RC into what becomes the final release, and making
> > it painful to backport first to 2.4.x and finally 2.4.n-rc branch
> > might help discourage disruptive backports, but there is no suggestion
> > yet of what is acceptable, either on such a frozen main branch or rc
> > branch.
> >
> > In fact our policy reflects that two competing release candidates is
> > an entire valid and even expected situation, and any process that
> > further blocks this should be firmly rebuffed.
> >
> > As it reads so far the proposal is the way we do things, now
> > conserving subversion numbers as precious, if only to reenforce a
> > stake in the ground that version major and minor numbers are similarly
> > precious, (which they are not.)
> >
> > With the addition of freezing changes on 2.4.x branch for a time we
> > succeed in impeading progress towards version .next, while not
> > otherwise changing the status quo.
> >
> > What you suggest is yet another thing, based on forking the RC release
> > branch, and I haven't seen that accepted by the committee yet.
> >
> > On Thu, Apr 19, 2018, 16:43 David Zuelke  > > wrote:
> >
> > The main difference is that you have a release branch in which fixes
> > to bugs or regressions found during 2.4.x RCs can be made, while work
> > on 2.4.(x+1) can continue in the main 2.4 branch.
> >
> > Another benefit is that people who do automated builds (e.g. me) can
> > grep for "RC" in the version number and have it fetch from
> > https://dist.apache.org/repos/dist/dev/httpd/ instead.
> >
> > The changelogs are more readable as well, because it's obvious which
> > intermediary RC releases belong together. If you look at
> > https://archive.apache.org/dist/httpd/CHANGES_2.4, there is zero
> > indication that e.g. 2.4.31 was never released.
> >
> >
> > On Thu, Apr 19, 2018 at 10:53 PM, William A Rowe Jr
> > > wrote:
> > > What possible improvement occurs if there is no branch discipline
> > > on 2.4.x development? We just echoed effectively your proposal,
> > > using numbers rather than RC designations, and still .33 has this
> > > host of issues. With no release since .29, the branch was in this
> > > continuous state of flux between 2.4.30 and 2.4.33. Testing the
> > > 2.4.30 release did not result in a better, eventual 2.4.31, testing
> > > .31 didn't result in a better .32, and even the final .33 had
> > its own
> > > issues.
> > >
> > > This would have been precisely the same outcome between RC1
> > > and RC4, if the project doesn't keep the branch stable for even the
> > > week or months required to craft a successful release, and if that
> > > occurs on 2.4.x branch, that is an interruption of effort towards
> > > release n+1, frustrating contributors.
> > >
> > > Note 

Re: Start using RCs (Was: Re: So... when should we do 2.4.34? [WAS: Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)])

[Daniel Ruggeri]

I'm not sure where in the conversation to add this, but I do want to
point out a mechanical concern.


If we end up with API and feature freeze on branch 2.4, then we'd expect
to roll 2.6. Soon enough, we'll hit a situation where 2.6 (as a release
branch) can't get a feature or the next great thing because of a
required API incompatible change. We would then kick out 2.8, 2.10 and
so on and so forth. This seems like it would satisfy both the
keep-the-stuff-stable as well as the give-users-new-stuff "sides".


Mechanically, this can become tiresome for a volunteer as we work
through the STATUS ceremony for each of the branches. I remember that
being less than enjoyable when 2.0, 2.2 and 2.4 were all release
branches. I'm fearful of a future where we may have five branches and a
bugfix applicable to all (or worse... a security fix that would dictate
all should be released/disclosed at the same time).

-- 
Daniel Ruggeri

On 4/19/2018 7:17 PM, William A Rowe Jr wrote:
> I don't disagree with RC's entirely, or the mechanism you suggest, but
> that isn't what I read as proposed.
>
> Our issue is that every httpd must be distinguished, we won't ship
> four tarballs all claiming 2.4.34 (GA). Which one is the person
> complaining about? Are we back to SHA signature of the source tarball
> (that isn't even assuredly what they built the binary from?)
>
> We can decorate the rc in the versioning, but that isn't part of 2.4.x
> API, unless we swap out the "-dev" string tag for an "-rc#" value.
>
> It is not reasonable to lock a branch for an indeterminate length of
> time. Branching the RC into what becomes the final release, and making
> it painful to backport first to 2.4.x and finally 2.4.n-rc branch
> might help discourage disruptive backports, but there is no suggestion
> yet of what is acceptable, either on such a frozen main branch or rc
> branch.
>
> In fact our policy reflects that two competing release candidates is
> an entire valid and even expected situation, and any process that
> further blocks this should be firmly rebuffed.
>
> As it reads so far the proposal is the way we do things, now
> conserving subversion numbers as precious, if only to reenforce a
> stake in the ground that version major and minor numbers are similarly
> precious, (which they are not.)
>
> With the addition of freezing changes on 2.4.x branch for a time we
> succeed in impeading progress towards version .next, while not
> otherwise changing the status quo.
>
> What you suggest is yet another thing, based on forking the RC release
> branch, and I haven't seen that accepted by the committee yet.
>
> On Thu, Apr 19, 2018, 16:43 David Zuelke  > wrote:
>
> The main difference is that you have a release branch in which fixes
> to bugs or regressions found during 2.4.x RCs can be made, while work
> on 2.4.(x+1) can continue in the main 2.4 branch.
>
> Another benefit is that people who do automated builds (e.g. me) can
> grep for "RC" in the version number and have it fetch from
> https://dist.apache.org/repos/dist/dev/httpd/ instead.
>
> The changelogs are more readable as well, because it's obvious which
> intermediary RC releases belong together. If you look at
> https://archive.apache.org/dist/httpd/CHANGES_2.4, there is zero
> indication that e.g. 2.4.31 was never released.
>
>
> On Thu, Apr 19, 2018 at 10:53 PM, William A Rowe Jr
> > wrote:
> > What possible improvement occurs if there is no branch discipline
> > on 2.4.x development? We just echoed effectively your proposal,
> > using numbers rather than RC designations, and still .33 has this
> > host of issues. With no release since .29, the branch was in this
> > continuous state of flux between 2.4.30 and 2.4.33. Testing the
> > 2.4.30 release did not result in a better, eventual 2.4.31, testing
> > .31 didn't result in a better .32, and even the final .33 had
> its own
> > issues.
> >
> > This would have been precisely the same outcome between RC1
> > and RC4, if the project doesn't keep the branch stable for even the
> > week or months required to craft a successful release, and if that
> > occurs on 2.4.x branch, that is an interruption of effort towards
> > release n+1, frustrating contributors.
> >
> > Note we don't publish anything not approved by the PMC, so
> > there would be the same "vote" to release an RC.
> >
> > Net <-> net, this is the status quo which failed us so badly this
> > past winter (now with alphabetic characters!)
> >
> >
> > On Thu, Apr 19, 2018 at 10:51 AM, Jim Jagielski  > wrote:
> >> The idea is encouraging and fostering a broader test audience.
> >>
> >>
> >> On Apr 19, 2018, at 11:44 AM, William A Rowe Jr
> 

Re: So... when should we do 2.4.34? [WAS: Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)]

[Daniel Ruggeri]

I don't think Stefan is proposing that we mindlessly ship based on an
arbitrary set of dates. That'd be too corporate :-)

I think in the past you mentioned that we have had features/fixes/Good
Stuff(tm) that was probably *ready*... just not yet released. I do
recall a few times where there have been fixes in 2.4.x branch that I
was frustrated as a user (burdened with the knowledge of a developer)
for having to wait to hit a release... so my main objective with the
automation is to simply make it simple to do a release so we aren't
entirely dependent on you and Bill to get those fixes/features/whatever
it may be into our users' hands.

Referring to the other message I just sent. I want to stress the fact
that now, with a few commands and *some* manual work (mostly when there
are CVEs to address), anyone can do a release. There's value in this -
especially given the fact that I truly did pore through and intensively
examine/internalize the documentation yet STILL managed to burn a
version number due to rookie mistakes. The third option along with
status quo or automation would be for the next unfortunate committer to
stumble through the process themselves and, possibly, make the same
mistakes I did.

Besides, heck... as you say, this is Open Source. We're volunteers and
this is the kind of stuff I actually do like to work on. CI/CD and the
like are a Good Thing :thumbsup:


Specifically to your question, Stefan... you beat me to it. Since the
last time I executed the release, I put in a quarterly reminder on my
calendar to check STATUS and help shuffle a release out the door if
there are no showstoppers and if we had added anything that looked cool.
I plan to stick by that schedule and also try to keep a pulse on the
project for times where we should ship a release sooner.

-- 
Daniel Ruggeri

On 4/19/2018 8:06 AM, Jim Jagielski wrote:
> One of the great things about working on open source is that
> one is NOT burdened by schedules. That releases are done
> "when ready" not when some artificial schedule or some
> calendar date demands. Changing this mindset on httpd would
> be an extremely major change, IMO, from what's been at its
> heart since 1995. Some may say that that is a good thing,
> that we need to "get inline with the times"... I would disagree,
> especially if we still want to encourage new contributions,
> new contributors and new (volunteer) committers.
>
> I submit that "doing a release" is not one of the problems that should
> be a priority to "fix".
>
>> On Apr 19, 2018, at 1:24 AM, Stefan Eissing  
>> wrote:
>>
>> Daniel,
>>
>> would you find it feasible to make a 2.4 release every first Tuesday of the 
>> month? Other projects have excellent experiences with such release timings. 
>>
>> I‘d expect this would let us focus on the changes („one more week until 
>> release“), take off pressure („we can do it in the next release“), avoid 
>> meta discussions („is this a good time?“) and let us streamline the 
>> test/release work with changes in process/automation with a higher 
>> motivation.
>>
>> Again, this would be your burden and call until we have so much 
>> routine/automation that everyone can do it. So it needs to be your decision.
>>
>> Cheers, Stefan
>>
>>> Am 19.04.2018 um 00:43 schrieb Daniel Ruggeri :
>>>
>>> On 4/18/2018 10:58 AM, William A Rowe Jr wrote:
> The release cycle is hours, to the benefit of all interested. Be it a 
> blocking bug fixed or a nice feature implemented. These are mostly people 
> who do it for fun. Some even run large server clusters, so a „hobbyist“ 
> label does not apply.
 Hours, yes, but we've had a willing RM, who has automated even
 more of this than Jim or I had, and has a very hard time finding
 any target to point to. E.g. "ok, that looks like the right resolution
 to the last of the regressions... let's..." ... "...oh there are all these
 other shiny objects in STATUS... rock-n-roll!!!" 
 ...
>>> What's particularly interesting to me, as I follow the conversation in
>>> my usual lurker-mode, is that Bill hit the nail on the head with this
>>> observation. I was waiting for the dust to settle to run through the
>>> scripts again for another T and release vote... but am not totally
>>> sure if we're ready. (mea culpa: my brain melted as I tried to follow
>>> the merging discussion so instead started parsing for "Yep. We're good
>>> now.")
>>>
>>> My current read on the conversation is that we're happy (or maybe just
>>> content) with the SSL merging fixes and we should prep to ship 2.4.34 as
>>> a fix. Does anyone disagree?
>>>
>>> -- 
>>> Daniel Ruggeri
>>>

Re: Start using RCs (Was: Re: So... when should we do 2.4.34? [WAS: Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)])

[William A Rowe Jr]

I don't disagree with RC's entirely, or the mechanism you suggest, but that
isn't what I read as proposed.

Our issue is that every httpd must be distinguished, we won't ship four
tarballs all claiming 2.4.34 (GA). Which one is the person complaining
about? Are we back to SHA signature of the source tarball (that isn't even
assuredly what they built the binary from?)

We can decorate the rc in the versioning, but that isn't part of 2.4.x API,
unless we swap out the "-dev" string tag for an "-rc#" value.

It is not reasonable to lock a branch for an indeterminate length of time.
Branching the RC into what becomes the final release, and making it painful
to backport first to 2.4.x and finally 2.4.n-rc branch might help
discourage disruptive backports, but there is no suggestion yet of what is
acceptable, either on such a frozen main branch or rc branch.

In fact our policy reflects that two competing release candidates is an
entire valid and even expected situation, and any process that further
blocks this should be firmly rebuffed.

As it reads so far the proposal is the way we do things, now conserving
subversion numbers as precious, if only to reenforce a stake in the ground
that version major and minor numbers are similarly precious, (which they
are not.)

With the addition of freezing changes on 2.4.x branch for a time we succeed
in impeading progress towards version .next, while not otherwise changing
the status quo.

What you suggest is yet another thing, based on forking the RC release
branch, and I haven't seen that accepted by the committee yet.

On Thu, Apr 19, 2018, 16:43 David Zuelke  wrote:

> The main difference is that you have a release branch in which fixes
> to bugs or regressions found during 2.4.x RCs can be made, while work
> on 2.4.(x+1) can continue in the main 2.4 branch.
>
> Another benefit is that people who do automated builds (e.g. me) can
> grep for "RC" in the version number and have it fetch from
> https://dist.apache.org/repos/dist/dev/httpd/ instead.
>
> The changelogs are more readable as well, because it's obvious which
> intermediary RC releases belong together. If you look at
> https://archive.apache.org/dist/httpd/CHANGES_2.4, there is zero
> indication that e.g. 2.4.31 was never released.
>
>
> On Thu, Apr 19, 2018 at 10:53 PM, William A Rowe Jr 
> wrote:
> > What possible improvement occurs if there is no branch discipline
> > on 2.4.x development? We just echoed effectively your proposal,
> > using numbers rather than RC designations, and still .33 has this
> > host of issues. With no release since .29, the branch was in this
> > continuous state of flux between 2.4.30 and 2.4.33. Testing the
> > 2.4.30 release did not result in a better, eventual 2.4.31, testing
> > .31 didn't result in a better .32, and even the final .33 had its own
> > issues.
> >
> > This would have been precisely the same outcome between RC1
> > and RC4, if the project doesn't keep the branch stable for even the
> > week or months required to craft a successful release, and if that
> > occurs on 2.4.x branch, that is an interruption of effort towards
> > release n+1, frustrating contributors.
> >
> > Note we don't publish anything not approved by the PMC, so
> > there would be the same "vote" to release an RC.
> >
> > Net <-> net, this is the status quo which failed us so badly this
> > past winter (now with alphabetic characters!)
> >
> >
> > On Thu, Apr 19, 2018 at 10:51 AM, Jim Jagielski  wrote:
> >> The idea is encouraging and fostering a broader test audience.
> >>
> >>
> >> On Apr 19, 2018, at 11:44 AM, William A Rowe Jr 
> wrote:
> >>
> >> Unless I misunderstand...
> >>
> >> 2.4.30-RC1 (rejected)
> >> 2.4.30-RC2 (our .31, rejected)
> >> 2.4.30-RC3 (our .32, rejected)
> >> 2.4.30-RC4 -> 2.4.30 GA (our 2.4.33 release)
> >>
> >> With all the associated changes in between, no actual change in branch
> >> management, scope, feature creep, etc?
> >>
> >> This sounds like dressing up the status quo with different labels.
> >>
> >>
> >>
> >> On Thu, Apr 19, 2018, 10:37 Jim Jagielski  wrote:
> >>>
> >>>
> >>>
> >>> > On Apr 19, 2018, at 11:26 AM, William A Rowe Jr  >
> >>> > wrote:
> >>> >
> >>> > On Thu, Apr 19, 2018 at 10:11 AM, Jim Jagielski 
> wrote:
> >>> >> With all this in mind, should we try to set things up so that the
> >>> >> next release cycle uses the concept of RCs?
> >>> >>
> >>> >> If so, and if people like, I can come up with a baseline
> >>> >> proposal on the process for us to debate and come to
> >>> >> some consensus on.
> >>> >
> >>> > Would you define an RC? What changes are allowable in that branch?
> >>>
> >>>
> >>> The idea is that right now we take an existing state in SVN
> >>> and tag it as, for example, 2.4.121. We then vote on that tag
> >>> and the artifacts released from that tag. No work is done on
> >>> the 2.4 branch 

Re: So... when should we do 2.4.34? [WAS: Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)]

[Daniel Ruggeri]

On 4/19/2018 7:55 AM, Eric Covener wrote:
>> Again, this would be your burden and call until we have so much 
>> routine/automation that everyone can do it. So it needs to be your decision.
> No offense intended here, but I did not really see as many html /
> script / process updates as I had expected. I was hoping the new eyes
> would get some of this stuff more explicit so it wasn't such a
> mystery.

Totally my fault. I just pushed the last bit of the scripts (I had set a
TODO to do it after we had reviewed in on private@, but failed to
complete the TODO). I still owe doc updates AND the promised docker
image to make the endeavor completely trivial but basically, I've
distilled the process down to this set of commands from the tools directory:

unset DISPLAY
TAG="2.4.33"
./tag.sh 2.4.x $TAG /tmp/foo
./release.sh --latestapxxx --tag $TAG '' httpd-2.4 $TAG
'drugg...@primary.net'
./push.sh . $TAG dev
#Wait for vote
./push.sh . $TAG dist
#Wait for mirrors
./announce.sh $TAG 'drugg...@apache.org' 'Daniel Ruggeri'

The only hangup so far is when we merge in security fixes right before
announce - if the patch in the private repo doesn't merge into CHANGES
cleanly, we end up with confusing CHANGES entries. This can be resolved
by a manual check or even an immediate fix after the script is run.
There are also a few XXX and TODO sections that could be addressed with
some more sophisticated automation.

-- 
Daniel Ruggeri

Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)

[David Zuelke]

On Thu, Apr 19, 2018 at 11:07 PM, Mark Blackman  wrote:
>
>
>> On 19 Apr 2018, at 21:35, David Zuelke  wrote:
>>
>> I'm not saying no directives should ever be added in point releases or
>> anything, but the constant backporting of *features* to 2.4 has
>> contributed to the relatively high number of regressions, and to a
>> lack of progress on 2.6/3.0, because, well, if anything can be put
>> into 2.4.next, why bother?
>>
>> David
>
> What’s the rule for *features*?

That remains to be defined. Generally, I'd say anything that doesn't
correct existing functionality, or anything that changes defaults, or
anything that changes behavior with existing settings, is a
feature/break/change and not a fix, so would belong in 2.next.0.

More or less the Semver approach, essentially.

See e.g. https://wiki.php.net/rfc/releaseprocess

David

Re: Start using RCs (Was: Re: So... when should we do 2.4.34? [WAS: Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)])

[David Zuelke]

The main difference is that you have a release branch in which fixes
to bugs or regressions found during 2.4.x RCs can be made, while work
on 2.4.(x+1) can continue in the main 2.4 branch.

Another benefit is that people who do automated builds (e.g. me) can
grep for "RC" in the version number and have it fetch from
https://dist.apache.org/repos/dist/dev/httpd/ instead.

The changelogs are more readable as well, because it's obvious which
intermediary RC releases belong together. If you look at
https://archive.apache.org/dist/httpd/CHANGES_2.4, there is zero
indication that e.g. 2.4.31 was never released.


On Thu, Apr 19, 2018 at 10:53 PM, William A Rowe Jr  wrote:
> What possible improvement occurs if there is no branch discipline
> on 2.4.x development? We just echoed effectively your proposal,
> using numbers rather than RC designations, and still .33 has this
> host of issues. With no release since .29, the branch was in this
> continuous state of flux between 2.4.30 and 2.4.33. Testing the
> 2.4.30 release did not result in a better, eventual 2.4.31, testing
> .31 didn't result in a better .32, and even the final .33 had its own
> issues.
>
> This would have been precisely the same outcome between RC1
> and RC4, if the project doesn't keep the branch stable for even the
> week or months required to craft a successful release, and if that
> occurs on 2.4.x branch, that is an interruption of effort towards
> release n+1, frustrating contributors.
>
> Note we don't publish anything not approved by the PMC, so
> there would be the same "vote" to release an RC.
>
> Net <-> net, this is the status quo which failed us so badly this
> past winter (now with alphabetic characters!)
>
>
> On Thu, Apr 19, 2018 at 10:51 AM, Jim Jagielski  wrote:
>> The idea is encouraging and fostering a broader test audience.
>>
>>
>> On Apr 19, 2018, at 11:44 AM, William A Rowe Jr  wrote:
>>
>> Unless I misunderstand...
>>
>> 2.4.30-RC1 (rejected)
>> 2.4.30-RC2 (our .31, rejected)
>> 2.4.30-RC3 (our .32, rejected)
>> 2.4.30-RC4 -> 2.4.30 GA (our 2.4.33 release)
>>
>> With all the associated changes in between, no actual change in branch
>> management, scope, feature creep, etc?
>>
>> This sounds like dressing up the status quo with different labels.
>>
>>
>>
>> On Thu, Apr 19, 2018, 10:37 Jim Jagielski  wrote:
>>>
>>>
>>>
>>> > On Apr 19, 2018, at 11:26 AM, William A Rowe Jr 
>>> > wrote:
>>> >
>>> > On Thu, Apr 19, 2018 at 10:11 AM, Jim Jagielski  wrote:
>>> >> With all this in mind, should we try to set things up so that the
>>> >> next release cycle uses the concept of RCs?
>>> >>
>>> >> If so, and if people like, I can come up with a baseline
>>> >> proposal on the process for us to debate and come to
>>> >> some consensus on.
>>> >
>>> > Would you define an RC? What changes are allowable in that branch?
>>>
>>>
>>> The idea is that right now we take an existing state in SVN
>>> and tag it as, for example, 2.4.121. We then vote on that tag
>>> and the artifacts released from that tag. No work is done on
>>> the 2.4 branch while testing is done so that we ensure that
>>> the SVN rev on branch == the one for the tag
>>
>>
>> Not necessary to freeze; a tag can always be applied to an older rev.
>>
>>> Instead, we tag at 2.4.121-RC1. We do the exact same. If the
>>> vote does not pass, we continue on the 2.4 branch, fix the
>>> issues, and then tag a 2.4.121-RC2. Rinse and repeat.
>>>
>>> If the vote does pass we tag the branch, which is still == RC#
>>> at this stage, as 2.4.121 (ap_release.h mods included). We
>>> still even at this stage test and vote on the release as we have
>>> for decades. If it passes, we release. If not, for some reason,
>>> we have burned the 2.4.121 release, bump to 2.4.122 and GOTO
>>> the above.
>>>
>>> This is the overall idea... more flesh to be added.
>>
>>

Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)

[Mark Blackman]

> On 19 Apr 2018, at 21:35, David Zuelke  wrote:
> 
> On Thu, Apr 19, 2018 at 8:25 PM, Jim Jagielski  wrote:
>> 
>> 
>>> On Apr 19, 2018, at 11:55 AM, David Zuelke  wrote:
>>> 
>>> 
>>> I hate to break this to you, and I do not want to discredit the
>>> amazing work all the contributors here are doing, but httpd 2.4 is of
>>> miserable, miserable quality when it comes to breaks and regressions.
>>> 
>> 
>> Gee Thanks! That is an amazing compliment to be sure. I have
>> NO idea how ANYONE could take that in any way as discrediting
>> the work being done.
>> 
>> Sarcasm aside, could we do better? Yes. Can we do better? Yes.
>> Should we do better? Yes. Will we do better? Yes.
>> 
>> BTW, you DID see how h2 actually came INTO httpd, didn't you??
> 
> Of course, but that's exactly my point. It was introduced not in
> 2.4.0, but in 2.4.17. Five "H2…" config directives are available in
> 2.4.18+ only, one in 2.4.19+, and three in 2.4.24+.
> 
> I'm not saying no directives should ever be added in point releases or
> anything, but the constant backporting of *features* to 2.4 has
> contributed to the relatively high number of regressions, and to a
> lack of progress on 2.6/3.0, because, well, if anything can be put
> into 2.4.next, why bother?
> 
> David

What’s the rule for *features*?

- Mark

Re: Start using RCs (Was: Re: So... when should we do 2.4.34? [WAS: Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)])

[William A Rowe Jr]

What possible improvement occurs if there is no branch discipline
on 2.4.x development? We just echoed effectively your proposal,
using numbers rather than RC designations, and still .33 has this
host of issues. With no release since .29, the branch was in this
continuous state of flux between 2.4.30 and 2.4.33. Testing the
2.4.30 release did not result in a better, eventual 2.4.31, testing
.31 didn't result in a better .32, and even the final .33 had its own
issues.

This would have been precisely the same outcome between RC1
and RC4, if the project doesn't keep the branch stable for even the
week or months required to craft a successful release, and if that
occurs on 2.4.x branch, that is an interruption of effort towards
release n+1, frustrating contributors.

Note we don't publish anything not approved by the PMC, so
there would be the same "vote" to release an RC.

Net <-> net, this is the status quo which failed us so badly this
past winter (now with alphabetic characters!)


On Thu, Apr 19, 2018 at 10:51 AM, Jim Jagielski  wrote:
> The idea is encouraging and fostering a broader test audience.
>
>
> On Apr 19, 2018, at 11:44 AM, William A Rowe Jr  wrote:
>
> Unless I misunderstand...
>
> 2.4.30-RC1 (rejected)
> 2.4.30-RC2 (our .31, rejected)
> 2.4.30-RC3 (our .32, rejected)
> 2.4.30-RC4 -> 2.4.30 GA (our 2.4.33 release)
>
> With all the associated changes in between, no actual change in branch
> management, scope, feature creep, etc?
>
> This sounds like dressing up the status quo with different labels.
>
>
>
> On Thu, Apr 19, 2018, 10:37 Jim Jagielski  wrote:
>>
>>
>>
>> > On Apr 19, 2018, at 11:26 AM, William A Rowe Jr 
>> > wrote:
>> >
>> > On Thu, Apr 19, 2018 at 10:11 AM, Jim Jagielski  wrote:
>> >> With all this in mind, should we try to set things up so that the
>> >> next release cycle uses the concept of RCs?
>> >>
>> >> If so, and if people like, I can come up with a baseline
>> >> proposal on the process for us to debate and come to
>> >> some consensus on.
>> >
>> > Would you define an RC? What changes are allowable in that branch?
>>
>>
>> The idea is that right now we take an existing state in SVN
>> and tag it as, for example, 2.4.121. We then vote on that tag
>> and the artifacts released from that tag. No work is done on
>> the 2.4 branch while testing is done so that we ensure that
>> the SVN rev on branch == the one for the tag
>
>
> Not necessary to freeze; a tag can always be applied to an older rev.
>
>> Instead, we tag at 2.4.121-RC1. We do the exact same. If the
>> vote does not pass, we continue on the 2.4 branch, fix the
>> issues, and then tag a 2.4.121-RC2. Rinse and repeat.
>>
>> If the vote does pass we tag the branch, which is still == RC#
>> at this stage, as 2.4.121 (ap_release.h mods included). We
>> still even at this stage test and vote on the release as we have
>> for decades. If it passes, we release. If not, for some reason,
>> we have burned the 2.4.121 release, bump to 2.4.122 and GOTO
>> the above.
>>
>> This is the overall idea... more flesh to be added.
>
>

Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)

[David Zuelke]

On Thu, Apr 19, 2018 at 8:25 PM, Jim Jagielski  wrote:
>
>
>> On Apr 19, 2018, at 11:55 AM, David Zuelke  wrote:
>>
>>
>> I hate to break this to you, and I do not want to discredit the
>> amazing work all the contributors here are doing, but httpd 2.4 is of
>> miserable, miserable quality when it comes to breaks and regressions.
>>
>
> Gee Thanks! That is an amazing compliment to be sure. I have
> NO idea how ANYONE could take that in any way as discrediting
> the work being done.
>
> Sarcasm aside, could we do better? Yes. Can we do better? Yes.
> Should we do better? Yes. Will we do better? Yes.
>
> BTW, you DID see how h2 actually came INTO httpd, didn't you??

Of course, but that's exactly my point. It was introduced not in
2.4.0, but in 2.4.17. Five "H2…" config directives are available in
2.4.18+ only, one in 2.4.19+, and three in 2.4.24+.

I'm not saying no directives should ever be added in point releases or
anything, but the constant backporting of *features* to 2.4 has
contributed to the relatively high number of regressions, and to a
lack of progress on 2.6/3.0, because, well, if anything can be put
into 2.4.next, why bother?

David

Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)

[William A Rowe Jr]

All informative feedback is welcome on this /discussion/ thread.

Jim, again, stop. Bullying list watchers with negative feedback
into silence is a CoC violation.

David, thank you for your detailed feedback. We are reading,
whether the feedback is warmly received or not.



On Thu, Apr 19, 2018 at 1:25 PM, Jim Jagielski  wrote:
>
>
>> On Apr 19, 2018, at 11:55 AM, David Zuelke  wrote:
>>
>>
>> I hate to break this to you, and I do not want to discredit the
>> amazing work all the contributors here are doing, but httpd 2.4 is of
>> miserable, miserable quality when it comes to breaks and regressions.
>>
>
> Gee Thanks! That is an amazing compliment to be sure. I have
> NO idea how ANYONE could take that in any way as discrediting
> the work being done.
>
> Sarcasm aside, could we do better? Yes. Can we do better? Yes.
> Should we do better? Yes. Will we do better? Yes.
>
> BTW, you DID see how h2 actually came INTO httpd, didn't you??
>

Re: Start using RCs (Was: Re: So... when should we do 2.4.34? [WAS: Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)])

[David Zuelke]

Yup, that's exactly it. Have a release branch, iterate there, and in
the meantime, work in the version series branch can continue. That
brings one huge benefit over the current model already: no freezes
necessary, no potential additional breaks after a "burned" version.

On Thu, Apr 19, 2018 at 9:19 PM, Rainer Jung  wrote:
> Am 19.04.2018 um 17:37 schrieb Jim Jagielski:
>>
>>
>>
>>> On Apr 19, 2018, at 11:26 AM, William A Rowe Jr 
>>> wrote:
>>>
>>> On Thu, Apr 19, 2018 at 10:11 AM, Jim Jagielski  wrote:

 With all this in mind, should we try to set things up so that the
 next release cycle uses the concept of RCs?

 If so, and if people like, I can come up with a baseline
 proposal on the process for us to debate and come to
 some consensus on.
>>>
>>>
>>> Would you define an RC? What changes are allowable in that branch?
>>
>>
>>
>> The idea is that right now we take an existing state in SVN
>> and tag it as, for example, 2.4.121. We then vote on that tag
>> and the artifacts released from that tag. No work is done on
>> the 2.4 branch while testing is done so that we ensure that
>> the SVN rev on branch == the one for the tag.
>>
>> Instead, we tag at 2.4.121-RC1. We do the exact same. If the
>> vote does not pass, we continue on the 2.4 branch, fix the
>> issues, and then tag a 2.4.121-RC2. Rinse and repeat.
>>
>> If the vote does pass we tag the branch, which is still == RC#
>> at this stage, as 2.4.121 (ap_release.h mods included). We
>> still even at this stage test and vote on the release as we have
>> for decades. If it passes, we release. If not, for some reason,
>> we have burned the 2.4.121 release, bump to 2.4.122 and GOTO
>> the above.
>>
>> This is the overall idea... more flesh to be added.
>
>
> IMHO we should aim at keeping the RC phase as stable as possible and focuse
> on the code from which we tagged RC1 and which we actually want to release.
> So after RC1 there should be only fixes for regressions, no other fixes or
> backports.
>
> Other projects for example branch at the start of the release process (using
> your version example 2.4.121 here):
>
> - branch 2.4.x as a 2.4.121 branch
> - tag 2.4.121-RC1 on that branch
> - allow for a week or two (?) of public testing
> - fix incoming regressions
>   - patches go via trunk to 2.4.x to 2.4.121 branch
> - cut new RCs if previous RC needed fixes
> - create final release tag from last RC plus some script driven version
> adjustments
> - do final release vote
>
> While we are doing RCs backport and other fixing work could proceed on the
> 2.4.x branch, because the RCs are created from the version branch.
>
> ... 2c ...
>
> Rainer

Re: Start using RCs (Was: Re: So... when should we do 2.4.34? [WAS: Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)])

[Rainer Jung]

Am 19.04.2018 um 17:37 schrieb Jim Jagielski:




On Apr 19, 2018, at 11:26 AM, William A Rowe Jr  wrote:

On Thu, Apr 19, 2018 at 10:11 AM, Jim Jagielski  wrote:

With all this in mind, should we try to set things up so that the
next release cycle uses the concept of RCs?

If so, and if people like, I can come up with a baseline
proposal on the process for us to debate and come to
some consensus on.


Would you define an RC? What changes are allowable in that branch?



The idea is that right now we take an existing state in SVN
and tag it as, for example, 2.4.121. We then vote on that tag
and the artifacts released from that tag. No work is done on
the 2.4 branch while testing is done so that we ensure that
the SVN rev on branch == the one for the tag.

Instead, we tag at 2.4.121-RC1. We do the exact same. If the
vote does not pass, we continue on the 2.4 branch, fix the
issues, and then tag a 2.4.121-RC2. Rinse and repeat.

If the vote does pass we tag the branch, which is still == RC#
at this stage, as 2.4.121 (ap_release.h mods included). We
still even at this stage test and vote on the release as we have
for decades. If it passes, we release. If not, for some reason,
we have burned the 2.4.121 release, bump to 2.4.122 and GOTO
the above.

This is the overall idea... more flesh to be added.


IMHO we should aim at keeping the RC phase as stable as possible and 
focuse on the code from which we tagged RC1 and which we actually want 
to release. So after RC1 there should be only fixes for regressions, no 
other fixes or backports.


Other projects for example branch at the start of the release process 
(using your version example 2.4.121 here):


- branch 2.4.x as a 2.4.121 branch
- tag 2.4.121-RC1 on that branch
- allow for a week or two (?) of public testing
- fix incoming regressions
  - patches go via trunk to 2.4.x to 2.4.121 branch
- cut new RCs if previous RC needed fixes
- create final release tag from last RC plus some script driven version 
adjustments

- do final release vote

While we are doing RCs backport and other fixing work could proceed on 
the 2.4.x branch, because the RCs are created from the version branch.


... 2c ...

Rainer

Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)

[Jim Jagielski]

> On Apr 19, 2018, at 11:55 AM, David Zuelke  wrote:
> 
> 
> I hate to break this to you, and I do not want to discredit the
> amazing work all the contributors here are doing, but httpd 2.4 is of
> miserable, miserable quality when it comes to breaks and regressions.
> 

Gee Thanks! That is an amazing compliment to be sure. I have
NO idea how ANYONE could take that in any way as discrediting
the work being done.

Sarcasm aside, could we do better? Yes. Can we do better? Yes.
Should we do better? Yes. Will we do better? Yes.

BTW, you DID see how h2 actually came INTO httpd, didn't you??

Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)

[David Zuelke]

On Wed, Apr 18, 2018 at 8:01 AM, Stefan Eissing
 wrote:
>
>
>> Am 17.04.2018 um 19:18 schrieb William A Rowe Jr :
>>
>>> The architecture of v2.4 has been very stable, the need for breaking 
>>> changes has been largely non existent, and the focus since 2011 has been to 
>>> get changes backported to v2.4 instead.
>>>
>>> To distill this down to raw numbers, there have been 1546 discrete 
>>> backports (my simple grep of CHANGES) since 2011 - which has provided an 
>>> enormous amount of enhancement for the collective community’s scrutiny.
>>
>> And the corresponding number of regressions and behavior changes. None
>> of these have enjoyed an "RC" or "beta", whatever one calls it, to
>> validate before adoption - other than our claim of "best httpd yet".
>> It has been an entirely new kitchen sink on every subversion release.
>
> All my substantial functional additions had beta releases for months before 
> being voted into the 2.4.x branch. With binary beta packages available for 
> several platforms by several supporters.
>
> William, this painting our world a dark and miserable place is coming back 
> every few months. It is a disservice to the people who contribute changes 
> here.

I hate to break this to you, and I do not want to discredit the
amazing work all the contributors here are doing, but httpd 2.4 is of
miserable, miserable quality when it comes to breaks and regressions.

I maintain the PHP/Apache/Nginx infrastructure at Heroku, and I was
able to use the following httpd releases only in the last ~2.5 years:

- 2.4.16
- 2.4.18
- 2.4.20
- 2.4.29
 -2.4.33

Mostly because of regressions around mod_proxy(_fcgi), REDIRECT_URL, whatever.

This is not any person's fault. This is the fault of the process. The
process can be repaired: bugfixes only in 2.4.x, do RC cycles for
bugfix releases as well (that alone makes the changelog look a lot
less confusing, which is useful for the project's image, see also the
Nginx marketing/FUD discussion in the other thread), and start testing
new features in modules first.

It makes such little sense to land h2 support in 2.4.something, as
opposed to having it as an official "brand new, try it out" subproject
first, and then bundle it with 2.6.

Speaking of which, I'd also suggest dropping this odd/even number
meaning experimental/stable versioning scheme, since it only
aggravates the problem: never-ending experiments that go stale, maybe
even get half backported, and meanwhile are subconsciously perceived
as one more hurdle towards a next bigger release.

Really, I'd suggest taking a close look at the PHP release cycle, with
their schedules, their RFC policies, everything. As I said in that
other thread, the PHP project was in exactly the same spot a few years
ago and they have pulled themselves out of that mess with amazing
results.

I am also happy to make introductions to release managers and
maintainers there. Heck I am betting some of them would happily serve
as tutors for the httpd project ;) I'm certainly willing to help too.
But IMO you need a clean cut and shake up the entire process, not just
a little, because otherwise you won't get rid of some of the old
habits that have been plaguing the project.

David

Re: Start using RCs (Was: Re: So... when should we do 2.4.34? [WAS: Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)])

[Jim Jagielski]

The idea is encouraging and fostering a broader test audience.

> On Apr 19, 2018, at 11:44 AM, William A Rowe Jr  wrote:
> 
> Unless I misunderstand...
> 
> 2.4.30-RC1 (rejected)
> 2.4.30-RC2 (our .31, rejected)
> 2.4.30-RC3 (our .32, rejected)
> 2.4.30-RC4 -> 2.4.30 GA (our 2.4.33 release)
> 
> With all the associated changes in between, no actual change in branch 
> management, scope, feature creep, etc?
> 
> This sounds like dressing up the status quo with different labels.
> 
> 
> 
> On Thu, Apr 19, 2018, 10:37 Jim Jagielski  > wrote:
> 
> 
> > On Apr 19, 2018, at 11:26 AM, William A Rowe Jr  > > wrote:
> > 
> > On Thu, Apr 19, 2018 at 10:11 AM, Jim Jagielski  > > wrote:
> >> With all this in mind, should we try to set things up so that the
> >> next release cycle uses the concept of RCs?
> >> 
> >> If so, and if people like, I can come up with a baseline
> >> proposal on the process for us to debate and come to
> >> some consensus on.
> > 
> > Would you define an RC? What changes are allowable in that branch?
> 
> 
> The idea is that right now we take an existing state in SVN
> and tag it as, for example, 2.4.121. We then vote on that tag
> and the artifacts released from that tag. No work is done on
> the 2.4 branch while testing is done so that we ensure that
> the SVN rev on branch == the one for the tag
> 
> Not necessary to freeze; a tag can always be applied to an older rev.
> 
> Instead, we tag at 2.4.121-RC1. We do the exact same. If the
> vote does not pass, we continue on the 2.4 branch, fix the
> issues, and then tag a 2.4.121-RC2. Rinse and repeat.
> 
> If the vote does pass we tag the branch, which is still == RC#
> at this stage, as 2.4.121 (ap_release.h mods included). We
> still even at this stage test and vote on the release as we have
> for decades. If it passes, we release. If not, for some reason,
> we have burned the 2.4.121 release, bump to 2.4.122 and GOTO
> the above.
> 
> This is the overall idea... more flesh to be added.

Re: Start using RCs (Was: Re: So... when should we do 2.4.34? [WAS: Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)])

[William A Rowe Jr]

Unless I misunderstand...

2.4.30-RC1 (rejected)
2.4.30-RC2 (our .31, rejected)
2.4.30-RC3 (our .32, rejected)
2.4.30-RC4 -> 2.4.30 GA (our 2.4.33 release)

With all the associated changes in between, no actual change in branch
management, scope, feature creep, etc?

This sounds like dressing up the status quo with different labels.



On Thu, Apr 19, 2018, 10:37 Jim Jagielski  wrote:

>
>
> > On Apr 19, 2018, at 11:26 AM, William A Rowe Jr 
> wrote:
> >
> > On Thu, Apr 19, 2018 at 10:11 AM, Jim Jagielski  wrote:
> >> With all this in mind, should we try to set things up so that the
> >> next release cycle uses the concept of RCs?
> >>
> >> If so, and if people like, I can come up with a baseline
> >> proposal on the process for us to debate and come to
> >> some consensus on.
> >
> > Would you define an RC? What changes are allowable in that branch?
>
>
> The idea is that right now we take an existing state in SVN
> and tag it as, for example, 2.4.121. We then vote on that tag
> and the artifacts released from that tag. No work is done on
> the 2.4 branch while testing is done so that we ensure that
> the SVN rev on branch == the one for the tag
>

Not necessary to freeze; a tag can always be applied to an older rev.

Instead, we tag at 2.4.121-RC1. We do the exact same. If the
> vote does not pass, we continue on the 2.4 branch, fix the
> issues, and then tag a 2.4.121-RC2. Rinse and repeat.
>
> If the vote does pass we tag the branch, which is still == RC#
> at this stage, as 2.4.121 (ap_release.h mods included). We
> still even at this stage test and vote on the release as we have
> for decades. If it passes, we release. If not, for some reason,
> we have burned the 2.4.121 release, bump to 2.4.122 and GOTO
> the above.
>
> This is the overall idea... more flesh to be added.

Re: Start using RCs (Was: Re: So... when should we do 2.4.34? [WAS: Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)])

[Jim Jagielski]

> On Apr 19, 2018, at 11:26 AM, William A Rowe Jr  wrote:
> 
> On Thu, Apr 19, 2018 at 10:11 AM, Jim Jagielski  wrote:
>> With all this in mind, should we try to set things up so that the
>> next release cycle uses the concept of RCs?
>> 
>> If so, and if people like, I can come up with a baseline
>> proposal on the process for us to debate and come to
>> some consensus on.
> 
> Would you define an RC? What changes are allowable in that branch?


The idea is that right now we take an existing state in SVN
and tag it as, for example, 2.4.121. We then vote on that tag
and the artifacts released from that tag. No work is done on
the 2.4 branch while testing is done so that we ensure that
the SVN rev on branch == the one for the tag.

Instead, we tag at 2.4.121-RC1. We do the exact same. If the
vote does not pass, we continue on the 2.4 branch, fix the
issues, and then tag a 2.4.121-RC2. Rinse and repeat.

If the vote does pass we tag the branch, which is still == RC#
at this stage, as 2.4.121 (ap_release.h mods included). We
still even at this stage test and vote on the release as we have
for decades. If it passes, we release. If not, for some reason,
we have burned the 2.4.121 release, bump to 2.4.122 and GOTO
the above.

This is the overall idea... more flesh to be added.

Re: Start using RCs (Was: Re: So... when should we do 2.4.34? [WAS: Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)])

[William A Rowe Jr]

On Thu, Apr 19, 2018 at 10:11 AM, Jim Jagielski  wrote:
> With all this in mind, should we try to set things up so that the
> next release cycle uses the concept of RCs?
>
> If so, and if people like, I can come up with a baseline
> proposal on the process for us to debate and come to
> some consensus on.

Would you define an RC? What changes are allowable in that branch?

Re: So... when should we do 2.4.34? [WAS: Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)]

[William A Rowe Jr]

On Thu, Apr 19, 2018 at 10:08 AM, Jim Jagielski  wrote:
>
>> On Apr 19, 2018, at 10:47 AM, William A Rowe Jr  wrote:
>>
>>  and BFDL/NIH-tier levels of "we don't do that, we do things this way... my 
>> way or the highway."
>
> That is not quite true nor fair.
>
> It does not take a BFDL/NIH-er, for example, to say "We don't
> release s/w under the GPLv3", nor if someone sez that, does
> that make them a BDFL. Nor does anyone have the ability
> to have anything like "my way or the highway" even stick.

We aren't discussing GPLv3 or any other ASF-wide policy here.
We are discussing how the HTTP Server project versions and
releases software for the public good. There are dozens of ways
we can accomplish that, all of which fit into ASF policy.

We operate by consensus. Which means, if any PMC member
is unwilling to accept change to our release or versioning, we are
stuck with status quo. Any time the argument against a change
becomes "that isn't how we have done it", the argument needs
to be judged by the success of that status quo.

This entire thread is predicated with my belief that the status quo
processes, not the code base or committers, has problems. All
the proposals made in previous threads on the subject were shouted
down, and the project release management, measured in bugs
and regressions, still suffers all the same issues as it has for a
number of years.

Could you be kind enough to point out where you have proposed
or accepted a change to the release process that we can use as
a starting point of a dialog? We are obviously not talking past
one another anymore that the process stands to be improved.

Since you have objected to each of my prior proposals (and all
those presented by others), I'd like to find out if we can converge
on one of your proposals?

Start using RCs (Was: Re: So... when should we do 2.4.34? [WAS: Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)])

[Jim Jagielski]

With all this in mind, should we try to set things up so that the
next release cycle uses the concept of RCs?

If so, and if people like, I can come up with a baseline
proposal on the process for us to debate and come to
some consensus on.

Re: So... when should we do 2.4.34? [WAS: Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)]

[Jim Jagielski]

> On Apr 19, 2018, at 10:47 AM, William A Rowe Jr  wrote:
> 
>  and BFDL/NIH-tier levels of "we don't do that, we do things this way... my 
> way or the highway."
> 

That is not quite true nor fair.

It does not take a BFDL/NIH-er, for example, to say "We don't
release s/w under the GPLv3", nor if someone sez that, does
that make them a BDFL. Nor does anyone have the ability
to have anything like "my way or the highway" even stick.
People can say it, sure, but I don't think anyone has,
unless, of course, in cases like the GPLv3 example above,
but then, obviously, it's not "my/their" way at all. It
is *our* way.

Please don't make this personal.

Re: So... when should we do 2.4.34? [WAS: Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)]

[William A Rowe Jr]

On Thu, Apr 19, 2018, 09:32 Jim Jagielski  wrote:

>
>
> > On Apr 19, 2018, at 10:20 AM, Stefan Eissing <
> stefan.eiss...@greenbytes.de> wrote:
> >
> > Frankly, I think the current state of things does not work well. It
> seems folly to say we should change nothing, only have more stable releases.
>
> No one is saying we change *nothing*. There just seems to be disagreement
> that what we should change is from "release when it's ready" to "release
> when the calendar sez
>

No suggested changes have met with acceptance, other than "more tests" and
some undefined "more frequent" releases.

I have several years of threads to dev@ of your's and a couple other's
objections to every substantive change proposed to the underlying process
by myself and others. All offered to achieve more consistent results for
our users to count on. Generally with the rationals of "that wouldn't be
fun" or "that isn't the ASF spirit" and BFDL/NIH-tier levels of "we don't
do that, we do things this way... my way or the highway."

That's why I introduced the subject differently, no perscriptions. What
would you suggest we *change* that would be fun/make people happy/produce
"the best available version of httpd" on a regular basis? I'm 100% with
Stefan, changing nothing except expectations of "doing better" is pure
folly.

While it is great we have fun, that can't continue at the expense of
other's enjoyment, and the expense of perpetually broken "stable" releases.

>

Re: So... when should we do 2.4.34? [WAS: Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)]

[Jim Jagielski]

> On Apr 19, 2018, at 10:20 AM, Stefan Eissing  
> wrote:
> 
> Frankly, I think the current state of things does not work well. It seems 
> folly to say we should change nothing, only have more stable releases.

No one is saying we change *nothing*. There just seems to be disagreement that 
what we should change is from "release when it's ready" to "release when the 
calendar sez"

Re: AW: So... when should we do 2.4.34? [WAS: Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)]

[Stefan Eissing]

Frankly, I think the current state of things does not work well. It seems folly 
to say we should change nothing, only have more stable releases.

Our current approach gave us around 6 months of „ship when it‘s ready“ and the 
quality is not what we want - as expressed by Jim, Bill and others.

Why a regular 2.4 release is in conflict with „ship when it‘s ready“ I do not 
understand. Surely, we only backport when it‘s ready? Where is this connected?

> Am 19.04.2018 um 15:41 schrieb Plüm, Rüdiger, Vodafone Group 
> <ruediger.pl...@vodafone.com>:
> 
> I am also more in the "ship when it is ready", then "ship when it is time" 
> boat.
> We probably could have some automated "nightlyies" which are not releases in 
> the ASF sense (as release requires voting),
> but only some sort of convenience transformation of an svn export / co that 
> creates a tar ball.
> But this only creates value if a broader audience tests them.
> But I guess most people that benefit from this easier processing of 
> "nightlyies" would only test them when they see that
> something interesting is contained for them.
> Which brings us back to "ship when its ready". OTOH " nightlyies" would add 
> testers that have different / their own
> criteria's on "when it is ready"
> 
> Regards
> 
> Rüdiger
> 
>> -Ursprüngliche Nachricht-
>> Von: Jim Jagielski <j...@jagunet.com>
>> Gesendet: Donnerstag, 19. April 2018 15:06
>> An: dev@httpd.apache.org
>> Betreff: Re: So... when should we do 2.4.34? [WAS: Re: Revisit
>> Versioning? (Was: 2.4.3x regression w/SSL vhost configs)]
>> 
>> One of the great things about working on open source is that
>> one is NOT burdened by schedules. That releases are done
>> "when ready" not when some artificial schedule or some
>> calendar date demands. Changing this mindset on httpd would
>> be an extremely major change, IMO, from what's been at its
>> heart since 1995. Some may say that that is a good thing,
>> that we need to "get inline with the times"... I would disagree,
>> especially if we still want to encourage new contributions,
>> new contributors and new (volunteer) committers.
>> 
>> I submit that "doing a release" is not one of the problems that should
>> be a priority to "fix".
>> 
>>> On Apr 19, 2018, at 1:24 AM, Stefan Eissing
>> <stefan.eiss...@greenbytes.de> wrote:
>>> 
>>> Daniel,
>>> 
>>> would you find it feasible to make a 2.4 release every first Tuesday
>> of the month? Other projects have excellent experiences with such
>> release timings.
>>> 
>>> I‘d expect this would let us focus on the changes („one more week
>> until release“), take off pressure („we can do it in the next release“),
>> avoid meta discussions („is this a good time?“) and let us streamline
>> the test/release work with changes in process/automation with a higher
>> motivation.
>>> 
>>> Again, this would be your burden and call until we have so much
>> routine/automation that everyone can do it. So it needs to be your
>> decision.
>>> 
>>> Cheers, Stefan
>>> 
>>>> Am 19.04.2018 um 00:43 schrieb Daniel Ruggeri <drugg...@primary.net>:
>>>> 
>>>> On 4/18/2018 10:58 AM, William A Rowe Jr wrote:
>>>>>> The release cycle is hours, to the benefit of all interested. Be it
>> a blocking bug fixed or a nice feature implemented. These are mostly
>> people who do it for fun. Some even run large server clusters, so a
>> „hobbyist“ label does not apply.
>>>>> Hours, yes, but we've had a willing RM, who has automated even
>>>>> more of this than Jim or I had, and has a very hard time finding
>>>>> any target to point to. E.g. "ok, that looks like the right
>> resolution
>>>>> to the last of the regressions... let's..." ... "...oh there are all
>> these
>>>>> other shiny objects in STATUS... rock-n-roll!!!"
>>>>> ...
>>>> 
>>>> What's particularly interesting to me, as I follow the conversation
>> in
>>>> my usual lurker-mode, is that Bill hit the nail on the head with this
>>>> observation. I was waiting for the dust to settle to run through the
>>>> scripts again for another T and release vote... but am not totally
>>>> sure if we're ready. (mea culpa: my brain melted as I tried to follow
>>>> the merging discussion so instead started parsing for "Yep. We're
>> good
>>>> now.")
>>>> 
>>>> My current read on the conversation is that we're happy (or maybe
>> just
>>>> content) with the SSL merging fixes and we should prep to ship 2.4.34
>> as
>>>> a fix. Does anyone disagree?
>>>> 
>>>> --
>>>> Daniel Ruggeri
>>>> 
>>> 
> 

Re: So... when should we do 2.4.34? [WAS: Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)]

[Luca Toscano]

2018-04-19 15:40 GMT+02:00 Eric Covener :

> On Thu, Apr 19, 2018 at 9:31 AM, Jim Jagielski  wrote:
> > I agree a case could be made for considering adding an RC stage
> > to our release process... it would require some additional tooling
> > and some sort of additions to ap_release.h but nothing insurmountable.
> >
> > That might be a nice small-and-easily-reversable step to address
> > some of the issues that we've been having lately.
>
> Including burning version numbers!
>

+1

AW: So... when should we do 2.4.34? [WAS: Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)]

[Plüm , Rüdiger , Vodafone Group]

I am also more in the "ship when it is ready", then "ship when it is time" boat.
We probably could have some automated "nightlyies" which are not releases in 
the ASF sense (as release requires voting),
but only some sort of convenience transformation of an svn export / co that 
creates a tar ball.
But this only creates value if a broader audience tests them.
But I guess most people that benefit from this easier processing of 
"nightlyies" would only test them when they see that
something interesting is contained for them.
Which brings us back to "ship when its ready". OTOH " nightlyies" would add 
testers that have different / their own
criteria's on "when it is ready"

Regards

Rüdiger

> -Ursprüngliche Nachricht-
> Von: Jim Jagielski <j...@jagunet.com>
> Gesendet: Donnerstag, 19. April 2018 15:06
> An: dev@httpd.apache.org
> Betreff: Re: So... when should we do 2.4.34? [WAS: Re: Revisit
> Versioning? (Was: 2.4.3x regression w/SSL vhost configs)]
> 
> One of the great things about working on open source is that
> one is NOT burdened by schedules. That releases are done
> "when ready" not when some artificial schedule or some
> calendar date demands. Changing this mindset on httpd would
> be an extremely major change, IMO, from what's been at its
> heart since 1995. Some may say that that is a good thing,
> that we need to "get inline with the times"... I would disagree,
> especially if we still want to encourage new contributions,
> new contributors and new (volunteer) committers.
> 
> I submit that "doing a release" is not one of the problems that should
> be a priority to "fix".
> 
> > On Apr 19, 2018, at 1:24 AM, Stefan Eissing
> <stefan.eiss...@greenbytes.de> wrote:
> >
> > Daniel,
> >
> > would you find it feasible to make a 2.4 release every first Tuesday
> of the month? Other projects have excellent experiences with such
> release timings.
> >
> > I‘d expect this would let us focus on the changes („one more week
> until release“), take off pressure („we can do it in the next release“),
> avoid meta discussions („is this a good time?“) and let us streamline
> the test/release work with changes in process/automation with a higher
> motivation.
> >
> > Again, this would be your burden and call until we have so much
> routine/automation that everyone can do it. So it needs to be your
> decision.
> >
> > Cheers, Stefan
> >
> >> Am 19.04.2018 um 00:43 schrieb Daniel Ruggeri <drugg...@primary.net>:
> >>
> >> On 4/18/2018 10:58 AM, William A Rowe Jr wrote:
> >>>> The release cycle is hours, to the benefit of all interested. Be it
> a blocking bug fixed or a nice feature implemented. These are mostly
> people who do it for fun. Some even run large server clusters, so a
> „hobbyist“ label does not apply.
> >>> Hours, yes, but we've had a willing RM, who has automated even
> >>> more of this than Jim or I had, and has a very hard time finding
> >>> any target to point to. E.g. "ok, that looks like the right
> resolution
> >>> to the last of the regressions... let's..." ... "...oh there are all
> these
> >>> other shiny objects in STATUS... rock-n-roll!!!"
> >>> ...
> >>
> >> What's particularly interesting to me, as I follow the conversation
> in
> >> my usual lurker-mode, is that Bill hit the nail on the head with this
> >> observation. I was waiting for the dust to settle to run through the
> >> scripts again for another T and release vote... but am not totally
> >> sure if we're ready. (mea culpa: my brain melted as I tried to follow
> >> the merging discussion so instead started parsing for "Yep. We're
> good
> >> now.")
> >>
> >> My current read on the conversation is that we're happy (or maybe
> just
> >> content) with the SSL merging fixes and we should prep to ship 2.4.34
> as
> >> a fix. Does anyone disagree?
> >>
> >> --
> >> Daniel Ruggeri
> >>
> >

Re: So... when should we do 2.4.34? [WAS: Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)]

[Eric Covener]

On Thu, Apr 19, 2018 at 9:31 AM, Jim Jagielski  wrote:
> I agree a case could be made for considering adding an RC stage
> to our release process... it would require some additional tooling
> and some sort of additions to ap_release.h but nothing insurmountable.
>
> That might be a nice small-and-easily-reversable step to address
> some of the issues that we've been having lately.

Including burning version numbers!

Re: So... when should we do 2.4.34? [WAS: Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)]

[Jim Jagielski]

I agree a case could be made for considering adding an RC stage
to our release process... it would require some additional tooling
and some sort of additions to ap_release.h but nothing insurmountable.

That might be a nice small-and-easily-reversable step to address
some of the issues that we've been having lately.

> On Apr 19, 2018, at 9:25 AM, Steffen  wrote:
> 
> ++1
> 
> The current versioning and times are fine for me. Only the vote time is too 
> short. At Apachelounge I have once in a while  RC’s from branches before 
> voting, so the community had more time to test.  Issues are then earlier 
> discovered.  
> 
> Distributors are free to have a RC any time when they think, looking at 
> changes in branches, it helps. 
> 
>> Op 19 apr. 2018 om 15:06 heeft Jim Jagielski  het volgende 
>> geschreven:
>> 
>> One of the great things about working on open source is that
>> one is NOT burdened by schedules. That releases are done
>> "when ready" not when some artificial schedule or some
>> calendar date demands. Changing this mindset on httpd would
>> be an extremely major change, IMO, from what's been at its
>> heart since 1995. Some may say that that is a good thing,
>> that we need to "get inline with the times"... I would disagree,
>> especially if we still want to encourage new contributions,
>> new contributors and new (volunteer) committers.
>> 
>> I submit that "doing a release" is not one of the problems that should
>> be a priority to "fix".
>> 
>>> On Apr 19, 2018, at 1:24 AM, Stefan Eissing  
>>> wrote:
>>> 
>>> Daniel,
>>> 
>>> would you find it feasible to make a 2.4 release every first Tuesday of the 
>>> month? Other projects have excellent experiences with such release timings. 
>>> 
>>> I‘d expect this would let us focus on the changes („one more week until 
>>> release“), take off pressure („we can do it in the next release“), avoid 
>>> meta discussions („is this a good time?“) and let us streamline the 
>>> test/release work with changes in process/automation with a higher 
>>> motivation.
>>> 
>>> Again, this would be your burden and call until we have so much 
>>> routine/automation that everyone can do it. So it needs to be your decision.
>>> 
>>> Cheers, Stefan
>>> 
 Am 19.04.2018 um 00:43 schrieb Daniel Ruggeri :
 
 On 4/18/2018 10:58 AM, William A Rowe Jr wrote:
>> The release cycle is hours, to the benefit of all interested. Be it a 
>> blocking bug fixed or a nice feature implemented. These are mostly 
>> people who do it for fun. Some even run large server clusters, so a 
>> „hobbyist“ label does not apply.
> Hours, yes, but we've had a willing RM, who has automated even
> more of this than Jim or I had, and has a very hard time finding
> any target to point to. E.g. "ok, that looks like the right resolution
> to the last of the regressions... let's..." ... "...oh there are all these
> other shiny objects in STATUS... rock-n-roll!!!" 
> ...
 
 What's particularly interesting to me, as I follow the conversation in
 my usual lurker-mode, is that Bill hit the nail on the head with this
 observation. I was waiting for the dust to settle to run through the
 scripts again for another T and release vote... but am not totally
 sure if we're ready. (mea culpa: my brain melted as I tried to follow
 the merging discussion so instead started parsing for "Yep. We're good
 now.")
 
 My current read on the conversation is that we're happy (or maybe just
 content) with the SSL merging fixes and we should prep to ship 2.4.34 as
 a fix. Does anyone disagree?
 
 -- 
 Daniel Ruggeri
 
>>> 
>> 
> 

Re: So... when should we do 2.4.34? [WAS: Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)]

[Steffen]

++1

The current versioning and times are fine for me. Only the vote time is too 
short. At Apachelounge I have once in a while  RC’s from branches before 
voting, so the community had more time to test.  Issues are then earlier 
discovered.  

Distributors are free to have a RC any time when they think, looking at changes 
in branches, it helps. 

> Op 19 apr. 2018 om 15:06 heeft Jim Jagielski  het volgende 
> geschreven:
> 
> One of the great things about working on open source is that
> one is NOT burdened by schedules. That releases are done
> "when ready" not when some artificial schedule or some
> calendar date demands. Changing this mindset on httpd would
> be an extremely major change, IMO, from what's been at its
> heart since 1995. Some may say that that is a good thing,
> that we need to "get inline with the times"... I would disagree,
> especially if we still want to encourage new contributions,
> new contributors and new (volunteer) committers.
> 
> I submit that "doing a release" is not one of the problems that should
> be a priority to "fix".
> 
>> On Apr 19, 2018, at 1:24 AM, Stefan Eissing  
>> wrote:
>> 
>> Daniel,
>> 
>> would you find it feasible to make a 2.4 release every first Tuesday of the 
>> month? Other projects have excellent experiences with such release timings. 
>> 
>> I‘d expect this would let us focus on the changes („one more week until 
>> release“), take off pressure („we can do it in the next release“), avoid 
>> meta discussions („is this a good time?“) and let us streamline the 
>> test/release work with changes in process/automation with a higher 
>> motivation.
>> 
>> Again, this would be your burden and call until we have so much 
>> routine/automation that everyone can do it. So it needs to be your decision.
>> 
>> Cheers, Stefan
>> 
>>> Am 19.04.2018 um 00:43 schrieb Daniel Ruggeri :
>>> 
>>> On 4/18/2018 10:58 AM, William A Rowe Jr wrote:
> The release cycle is hours, to the benefit of all interested. Be it a 
> blocking bug fixed or a nice feature implemented. These are mostly people 
> who do it for fun. Some even run large server clusters, so a „hobbyist“ 
> label does not apply.
 Hours, yes, but we've had a willing RM, who has automated even
 more of this than Jim or I had, and has a very hard time finding
 any target to point to. E.g. "ok, that looks like the right resolution
 to the last of the regressions... let's..." ... "...oh there are all these
 other shiny objects in STATUS... rock-n-roll!!!" 
 ...
>>> 
>>> What's particularly interesting to me, as I follow the conversation in
>>> my usual lurker-mode, is that Bill hit the nail on the head with this
>>> observation. I was waiting for the dust to settle to run through the
>>> scripts again for another T and release vote... but am not totally
>>> sure if we're ready. (mea culpa: my brain melted as I tried to follow
>>> the merging discussion so instead started parsing for "Yep. We're good
>>> now.")
>>> 
>>> My current read on the conversation is that we're happy (or maybe just
>>> content) with the SSL merging fixes and we should prep to ship 2.4.34 as
>>> a fix. Does anyone disagree?
>>> 
>>> -- 
>>> Daniel Ruggeri
>>> 
>> 
> 

Re: So... when should we do 2.4.34? [WAS: Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)]

[Jim Jagielski]

One of the great things about working on open source is that
one is NOT burdened by schedules. That releases are done
"when ready" not when some artificial schedule or some
calendar date demands. Changing this mindset on httpd would
be an extremely major change, IMO, from what's been at its
heart since 1995. Some may say that that is a good thing,
that we need to "get inline with the times"... I would disagree,
especially if we still want to encourage new contributions,
new contributors and new (volunteer) committers.

I submit that "doing a release" is not one of the problems that should
be a priority to "fix".

> On Apr 19, 2018, at 1:24 AM, Stefan Eissing  
> wrote:
> 
> Daniel,
> 
> would you find it feasible to make a 2.4 release every first Tuesday of the 
> month? Other projects have excellent experiences with such release timings. 
> 
> I‘d expect this would let us focus on the changes („one more week until 
> release“), take off pressure („we can do it in the next release“), avoid meta 
> discussions („is this a good time?“) and let us streamline the test/release 
> work with changes in process/automation with a higher motivation.
> 
> Again, this would be your burden and call until we have so much 
> routine/automation that everyone can do it. So it needs to be your decision.
> 
> Cheers, Stefan
> 
>> Am 19.04.2018 um 00:43 schrieb Daniel Ruggeri :
>> 
>> On 4/18/2018 10:58 AM, William A Rowe Jr wrote:
 The release cycle is hours, to the benefit of all interested. Be it a 
 blocking bug fixed or a nice feature implemented. These are mostly people 
 who do it for fun. Some even run large server clusters, so a „hobbyist“ 
 label does not apply.
>>> Hours, yes, but we've had a willing RM, who has automated even
>>> more of this than Jim or I had, and has a very hard time finding
>>> any target to point to. E.g. "ok, that looks like the right resolution
>>> to the last of the regressions... let's..." ... "...oh there are all these
>>> other shiny objects in STATUS... rock-n-roll!!!" 
>>> ...
>> 
>> What's particularly interesting to me, as I follow the conversation in
>> my usual lurker-mode, is that Bill hit the nail on the head with this
>> observation. I was waiting for the dust to settle to run through the
>> scripts again for another T and release vote... but am not totally
>> sure if we're ready. (mea culpa: my brain melted as I tried to follow
>> the merging discussion so instead started parsing for "Yep. We're good
>> now.")
>> 
>> My current read on the conversation is that we're happy (or maybe just
>> content) with the SSL merging fixes and we should prep to ship 2.4.34 as
>> a fix. Does anyone disagree?
>> 
>> -- 
>> Daniel Ruggeri
>> 
> 

Re: So... when should we do 2.4.34? [WAS: Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)]

[Eric Covener]

> Again, this would be your burden and call until we have so much 
> routine/automation that everyone can do it. So it needs to be your decision.

No offense intended here, but I did not really see as many html /
script / process updates as I had expected. I was hoping the new eyes
would get some of this stuff more explicit so it wasn't such a
mystery.

Re: So... when should we do 2.4.34? [WAS: Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)]

[Eric Covener]

On Wed, Apr 18, 2018 at 6:43 PM, Daniel Ruggeri  wrote:
> On 4/18/2018 10:58 AM, William A Rowe Jr wrote:
>>> The release cycle is hours, to the benefit of all interested. Be it a 
>>> blocking bug fixed or a nice feature implemented. These are mostly people 
>>> who do it for fun. Some even run large server clusters, so a „hobbyist“ 
>>> label does not apply.
>> Hours, yes, but we've had a willing RM, who has automated even
>> more of this than Jim or I had, and has a very hard time finding
>> any target to point to. E.g. "ok, that looks like the right resolution
>> to the last of the regressions... let's..." ... "...oh there are all these
>> other shiny objects in STATUS... rock-n-roll!!!"
>> ...
>
> What's particularly interesting to me, as I follow the conversation in
> my usual lurker-mode, is that Bill hit the nail on the head with this
> observation. I was waiting for the dust to settle to run through the
> scripts again for another T and release vote... but am not totally
> sure if we're ready. (mea culpa: my brain melted as I tried to follow
> the merging discussion so instead started parsing for "Yep. We're good
> now.")
>
> My current read on the conversation is that we're happy (or maybe just
> content) with the SSL merging fixes and we should prep to ship 2.4.34 as
> a fix. Does anyone disagree?

I am not sure we have that fix yet in 2.4.x yet.

I think we should call the proxy thing (62308) a showstopper and not
roll a release without it.

Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)

[Graham Leggett]

On 19 Apr 2018, at 11:57 AM, Joe Orton  wrote:

> Feel like I should drop 2c in here...
> 
> I'd be VERY happy to see more frequent "major" version bumps, i.e. 
> 2.4->2.6->2.8 or whatever which break backwards compat/ABI.  We have the 
> chance to break compat every ~6 months in Fedora so it's no problem 
> getting new code into the hands of users.

As an end user of the software I would hate that.

I love the fact that I can drop httpd v2.4.latest as published by ASF onto a 
RHEL machine and it “just works”. No recompiling modules to a new ABI, 
particularly large modules with large ecosystems, no mess, no fuss. No 
discovery that to get feature X I need to upgrade through two major versions 
along with the dependency hell that results to get there.

I get it that this convenience comes at a price - Redhat is doing work that I 
would otherwise do, but then that’s what I’m paying Redhat for.

That said - yes, we should work to release v2.6.x. Just not every six months.

Regards,
Graham
—



smime.p7s
Description: S/MIME cryptographic signature

Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)

[Joe Orton]

Feel like I should drop 2c in here...

I'd be VERY happy to see more frequent "major" version bumps, i.e. 
2.4->2.6->2.8 or whatever which break backwards compat/ABI.  We have the 
chance to break compat every ~6 months in Fedora so it's no problem 
getting new code into the hands of users.

I've spent much of my upstream time this year trying to get all RHEL7 
httpd features backported to 2.4.x and have only made it about 90% 
of the way (some big chunks like mod_systemd, suexec stuff remain); 
would love to not have to burn more time on backports because that stuff 
is in 2.6.0 already.

At the moment I think we have to accept that 2.4.x is going to be a bit 
unstable if we're trying to backport everything without *either* having 
good test coverage (which we don't) or having new code widely tested by 
users.

Regards, Joe

Re: So... when should we do 2.4.34? [WAS: Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)]

[Stefan Eissing]

Daniel,

would you find it feasible to make a 2.4 release every first Tuesday of the 
month? Other projects have excellent experiences with such release timings. 

I‘d expect this would let us focus on the changes („one more week until 
release“), take off pressure („we can do it in the next release“), avoid meta 
discussions („is this a good time?“) and let us streamline the test/release 
work with changes in process/automation with a higher motivation.

Again, this would be your burden and call until we have so much 
routine/automation that everyone can do it. So it needs to be your decision.

Cheers, Stefan

> Am 19.04.2018 um 00:43 schrieb Daniel Ruggeri :
> 
> On 4/18/2018 10:58 AM, William A Rowe Jr wrote:
>>> The release cycle is hours, to the benefit of all interested. Be it a 
>>> blocking bug fixed or a nice feature implemented. These are mostly people 
>>> who do it for fun. Some even run large server clusters, so a „hobbyist“ 
>>> label does not apply.
>> Hours, yes, but we've had a willing RM, who has automated even
>> more of this than Jim or I had, and has a very hard time finding
>> any target to point to. E.g. "ok, that looks like the right resolution
>> to the last of the regressions... let's..." ... "...oh there are all these
>> other shiny objects in STATUS... rock-n-roll!!!" 
>> ...
> 
> What's particularly interesting to me, as I follow the conversation in
> my usual lurker-mode, is that Bill hit the nail on the head with this
> observation. I was waiting for the dust to settle to run through the
> scripts again for another T and release vote... but am not totally
> sure if we're ready. (mea culpa: my brain melted as I tried to follow
> the merging discussion so instead started parsing for "Yep. We're good
> now.")
> 
> My current read on the conversation is that we're happy (or maybe just
> content) with the SSL merging fixes and we should prep to ship 2.4.34 as
> a fix. Does anyone disagree?
> 
> -- 
> Daniel Ruggeri
> 

Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)

[Daniel Ruggeri]

On 4/18/2018 1:34 PM, Alain Toussaint wrote:
>> As an aside - httpd has a —enable-layout option in configure that defines 
>> where things should go.
>> If you patch the following file how you want it and submit it to us, we can 
>> formally support LFS
>> out the box and you can remove the need for your patch:
>>
>> https://svn.apache.org/repos/asf/httpd/sandbox/replacelimit/config.layout
>>
>> Regards,
>> Graham
>> —
>>
> Great idea which I'll submit to the power that be.
>
> Alain

Minor correction to the URL for latest and greatest:
https://svn.apache.org/repos/asf/httpd/httpd/trunk/config.layout

As we love to say, "patches welcome!"

Feel free to just submit your diff here (since dev@ IS the power that be)

-- 
Daniel Ruggeri

So... when should we do 2.4.34? [WAS: Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)]

[Daniel Ruggeri]

On 4/18/2018 10:58 AM, William A Rowe Jr wrote:
>> The release cycle is hours, to the benefit of all interested. Be it a 
>> blocking bug fixed or a nice feature implemented. These are mostly people 
>> who do it for fun. Some even run large server clusters, so a „hobbyist“ 
>> label does not apply.
> Hours, yes, but we've had a willing RM, who has automated even
> more of this than Jim or I had, and has a very hard time finding
> any target to point to. E.g. "ok, that looks like the right resolution
> to the last of the regressions... let's..." ... "...oh there are all these
> other shiny objects in STATUS... rock-n-roll!!!" 
> ...

What's particularly interesting to me, as I follow the conversation in
my usual lurker-mode, is that Bill hit the nail on the head with this
observation. I was waiting for the dust to settle to run through the
scripts again for another T and release vote... but am not totally
sure if we're ready. (mea culpa: my brain melted as I tried to follow
the merging discussion so instead started parsing for "Yep. We're good
now.")

My current read on the conversation is that we're happy (or maybe just
content) with the SSL merging fixes and we should prep to ship 2.4.34 as
a fix. Does anyone disagree?

-- 
Daniel Ruggeri

Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)

[Alain Toussaint]

Le mercredi 18 avril 2018 à 11:41 +0200, Graham Leggett a écrit :
> On 17 Apr 2018, at 7:17 PM, Alain Toussaint  wrote:
> 
> > > No
> > > distribution (that I am aware of) ships something called Apache httpd 
> > > v2.4.29.
> > 
> > At LFS (linux from scratch), we're the exception confirming the rule of 
> > shipping v2.4.29 with
> > the
> > single patch of defining a preferred layout (the BLFS layout patch) in 
> > LFS/BLFS v8.2.
> > 
> > B/LFS-svn is shipping with v2.4.33 currently.
> > 
> > Alain (bug chaser for B/LFS and ALFS working toward editorship).
> 
> Looking at http://www.linuxfromscratch.org/blfs/view/svn/server/apache.html 
> it doesn’t appear that
> you’re shipping httpd at all, instead you’re directing people to get httpd 
> from the ASF, and are
> supplying a patch to make it work with LFS. Both of these activities are 
> entirely fine.

We do have mirrors for the cases where upstream changes deviate from the book's 
instruction and the
automated ALFS tool draw primarily from the mirrors before hitting upstream but 
peoples doing thing
by hands while reading the books do take the packages from upstream.

> As an aside - httpd has a —enable-layout option in configure that defines 
> where things should go.
> If you patch the following file how you want it and submit it to us, we can 
> formally support LFS
> out the box and you can remove the need for your patch:
> 
> https://svn.apache.org/repos/asf/httpd/sandbox/replacelimit/config.layout
> 
> Regards,
> Graham
> —
> 

Great idea which I'll submit to the power that be.

Alain

Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)

[Rainer Jung]

Am 18.04.2018 um 18:07 schrieb William A Rowe Jr:

On Wed, Apr 18, 2018 at 10:57 AM, Rainer Jung  wrote:


Since this thread was triggered by the mod_ssl config merging problems: I
think that was a case where a new feature was really nice, but to implement
it the needed changes where not not easy to understand in detail. Combined
with the complex behavior of mod_ssl w.r.t. config merging we ended up with
unexpected config merging problems. So that specific problem belongs to your
class 1 (I think both, the one detected by Mark Blackman and the one
received via Joe).

It is not a total surprise, that regressions - be it due to features, bug
fixing or refactoring - most often happen in mod_proxy or mod_ssl. These are
IMHO by far the most complex modules (together with the event MPM).
Unfortunately the same parts are very attractive for features, so we have
some need to touch them.


Keep in mind Windows has been broken on nearly every release,
recently in the core and mod_ssl build. So although I forked an SSL
regression thread, please don't read into this that they are primary
"culprits"... even core changes broke mod_security.


You are right, windows builds is also a fragile area. Not because the 
builds themselves are fragile, but many of us do not have them in their 
focus.



I am not blaming either proxy+ssl modules, nor their developers.

I'm raising process issues, not contribution or contributor issues.
Looking for a scheme to let contributors shine by putting our code
enhancements and major refactoring through a community review
process, which has been neglected for most of this decade.



All in all I'd prefer an attempt to have a faster moving 2.6 and a stable
backport branch 2.4 real soon.


Question, if 2.6 is moving "fast", and handled as 2.4 was, is there any
net benefit for better releases? What can we agree to in versioning
prior to 2.6.x-GA that will ease the process for contributors, external
module authors, distributors and users?


I would expect people picking up 2.6 to be in a better position doing 
frequent updates and having a bit higher tolerance for an occasional 
break as the downside of getting new features quickly.


So people could choose: stick to the slowly moving 2.4 branch (and get 
it from the enterprise distro vendor) or switch to the faster moving 2.6 
with the downside of a higher risk in regressions and using a different 
source for the binaries (or of course build themselves).


Regards,

Rainer

Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)

[William A Rowe Jr]

On Wed, Apr 18, 2018 at 10:57 AM, Rainer Jung  wrote:
>
> Since this thread was triggered by the mod_ssl config merging problems: I
> think that was a case where a new feature was really nice, but to implement
> it the needed changes where not not easy to understand in detail. Combined
> with the complex behavior of mod_ssl w.r.t. config merging we ended up with
> unexpected config merging problems. So that specific problem belongs to your
> class 1 (I think both, the one detected by Mark Blackman and the one
> received via Joe).
>
> It is not a total surprise, that regressions - be it due to features, bug
> fixing or refactoring - most often happen in mod_proxy or mod_ssl. These are
> IMHO by far the most complex modules (together with the event MPM).
> Unfortunately the same parts are very attractive for features, so we have
> some need to touch them.

Keep in mind Windows has been broken on nearly every release,
recently in the core and mod_ssl build. So although I forked an SSL
regression thread, please don't read into this that they are primary
"culprits"... even core changes broke mod_security.

I am not blaming either proxy+ssl modules, nor their developers.

I'm raising process issues, not contribution or contributor issues.
Looking for a scheme to let contributors shine by putting our code
enhancements and major refactoring through a community review
process, which has been neglected for most of this decade.


> All in all I'd prefer an attempt to have a faster moving 2.6 and a stable
> backport branch 2.4 real soon.

Question, if 2.6 is moving "fast", and handled as 2.4 was, is there any
net benefit for better releases? What can we agree to in versioning
prior to 2.6.x-GA that will ease the process for contributors, external
module authors, distributors and users?

Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)

[William A Rowe Jr]

On Wed, Apr 18, 2018 at 1:01 AM, Stefan Eissing
 wrote:
>
>> Am 17.04.2018 um 19:18 schrieb William A Rowe Jr :
>>
>>> On Tue, Apr 17, 2018 at 11:17 AM, Graham Leggett  wrote:
 On 17 Apr 2018, at 6:08 PM, William A Rowe Jr  wrote:

 No enhancement since 2011-12-19 has been presented for the collective
 community's scrutiny.
>>>
>>> Again, I’m not following.
>>>
>>> The architecture of v2.4 has been very stable, the need for breaking 
>>> changes has been largely non existent, and the focus since 2011 has been to 
>>> get changes backported to v2.4 instead.
>>>
>>> To distill this down to raw numbers, there have been 1546 discrete 
>>> backports (my simple grep of CHANGES) since 2011 - which has provided an 
>>> enormous amount of enhancement for the collective community’s scrutiny.
>>
>> And the corresponding number of regressions and behavior changes. None
>> of these have enjoyed an "RC" or "beta", whatever one calls it, to
>> validate before adoption - other than our claim of "best httpd yet".
>> It has been an entirely new kitchen sink on every subversion release.
>
> All my substantial functional additions had beta releases for months before 
> being voted into the 2.4.x branch. With binary beta packages available for 
> several platforms by several supporters.

Yes; this is an exception. But as you first encountered, the scope of
changes requires extensive rewiring of the hook processing phases
and the architecture of modules themselves.

You are in one instance (h2) spanning two worlds, one of very stable
API's, architecture and predictable release cadences (nghttp2), and "us".
Which makes your life easier, and more enjoyable?

Since we never see a beta of the collective work, we don't pick up the
various build problems (mod_md.h missing from make install on unix?
CMake ignorant of new files and paths?) Your underlying module
sources, built from apxs or similar I'm not worried about. The various
patches required to glue it together is what I worry about.

In most versioning schemes, these fundamental changes to the API,
behavior and existing modules would have been set off with another
version minor, or when redefining the module struct and existing hook
behavior, a version major update. There would have been at least one
beta of the collective work, issues would have been uncovered, then
we return to fixing up the smaller bugs that don't require refactoring.

> William, this painting our world a dark and miserable place is coming back 
> every few months. It is a disservice to the people who contribute changes 
> here.

If stating the plain facts of the state of our current release(s) continues
to be dark and miserable, that mirrors some disservice to the people
who are *trying* to consume our software.

I understand why you would say that from a recent PMC business
private post; I plan to share (and paint dark) that picture with the full
dev community, with some real improvements.

>>> You seem to be making a mountain out of a molehill, I just don’t see the 
>>> problem you’re trying to solve.
>>
>> You are welcome to attribute this concern any way you like, and be
>> satisfied with whatever yardstick you wish to measure it by. If you
>> interpret our users as desiring enhancement and not stability, then
>> those are the interests you should advocate. I'll leave this thread
>> alone for another week again to give them the opportunity to chime in.
>
> There are alternative ways to be creative and innovate than going through 
> this PMC into a semi annual release.

Exactly... that's why I started this thread without any prescription.
Let's hear them all, and agree to some. There are some overloaded
deeply-held beliefs here about the scarcity of version majors which
we first need to set aside, starting another thread on the raw data
from that exercise.

> Releasing a module (plus some small patches) on github opens ways for 
> collaboration with people who like Apache and new stuff. Distros like Debian 
> unstable and Fedora pick up stuff from there. PPAs for apt are made 
> available. Steffen offers Windows builts.

Yes; this is true of both subversion releases and major version releases.

Take a radically incompatible example; PCRE 8 seems to have some
perpetual life while PCRE 10 has very slow adoption, because old
stack optimizations no longer play in a world where stack corruption
exploits are trivial, and he deliberately made such things hard to do.
Still, the distributors ship PCRE 10 to be consumed alongside the
older cruftier choice.

> The release cycle is hours, to the benefit of all interested. Be it a 
> blocking bug fixed or a nice feature implemented. These are mostly people who 
> do it for fun. Some even run large server clusters, so a „hobbyist“ label 
> does not apply.

Hours, yes, but we've had a willing RM, who has automated even
more of this than Jim or I had, and has a 

Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)

[Rainer Jung]

Am 18.04.2018 um 15:07 schrieb Jim Jagielski:

There are, IMO at least, 3 types of "regression" that we should be
concerned about or that some people are concerned about:

   1. New features:
  Undoubtedly, new features will likely have bugs and
  no by adding new features we could be adding bugs
  which could be seen as a regression.

   2. Simple fixes:
  A simple fix causes a regression.

   3. Wholesale refactoring:
  IMO, this is the one which is the most problematic for
  us lately. We have seen several cases where a simple
  bug or a desire to "make this part of the code better"
  has resulted in huge amounts of code churn, major rewrites
  and major refactoring.

My PoV is that:

  o We need to continue to add new features. We must provide better
QA and testing.

  o We need to avoid our natural inclinations to look at "fixing
bugs" as an opportunity for major refactors. If people want
to major refactor, fine. But that stays in trunk. What is
important is that we patch the bug 1st. Premature re-factoring
is as bad as premature optimization.


Since this thread was triggered by the mod_ssl config merging problems: 
I think that was a case where a new feature was really nice, but to 
implement it the needed changes where not not easy to understand in 
detail. Combined with the complex behavior of mod_ssl w.r.t. config 
merging we ended up with unexpected config merging problems. So that 
specific problem belongs to your class 1 (I think both, the one detected 
by Mark Blackman and the one received via Joe).


It is not a total surprise, that regressions - be it due to features, 
bug fixing or refactoring - most often happen in mod_proxy or mod_ssl. 
These are IMHO by far the most complex modules (together with the event 
MPM). Unfortunately the same parts are very attractive for features, so 
we have some need to touch them.


When implementing a feature or fixing a bug it is often hard to decide 
where the border is crossed that makes a change a "major" refactoring or 
whether a changes still mostly helps to understand the code.


We do lack a robust procedure and resources for testing complex 
configurations and also testing on many platforms. I don't have a 
solution for that :(


More release branches would only help, if people would actually pick 
them up. So we would need to keep features in the higher branches at 
least for a noticeable time. Of course the enterprise distros would not 
pick the additional release branches up, but maybe if they have 
attractive features, they might get picked up by faster moving distros, 
inside container images and by 3rd-parties like Apache Lounge. We could 
even provide builds for some platforms as a voluntary service.


All in all I'd prefer an attempt to have a faster moving 2.6 and a 
stable backport branch 2.4 real soon.


Regards,

Rainer

Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)

[Jim Jagielski]

There are, IMO at least, 3 types of "regression" that we should be
concerned about or that some people are concerned about:

  1. New features:
 Undoubtedly, new features will likely have bugs and
 no by adding new features we could be adding bugs
 which could be seen as a regression.

  2. Simple fixes:
 A simple fix causes a regression.

  3. Wholesale refactoring:
 IMO, this is the one which is the most problematic for
 us lately. We have seen several cases where a simple
 bug or a desire to "make this part of the code better"
 has resulted in huge amounts of code churn, major rewrites
 and major refactoring.

My PoV is that:

 o We need to continue to add new features. We must provide better
   QA and testing.

 o We need to avoid our natural inclinations to look at "fixing
   bugs" as an opportunity for major refactors. If people want
   to major refactor, fine. But that stays in trunk. What is
   important is that we patch the bug 1st. Premature re-factoring
   is as bad as premature optimization.

Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)

[Graham Leggett]

On 17 Apr 2018, at 7:17 PM, Alain Toussaint  wrote:

>> No
>> distribution (that I am aware of) ships something called Apache httpd 
>> v2.4.29.
> 
> At LFS (linux from scratch), we're the exception confirming the rule of 
> shipping v2.4.29 with the
> single patch of defining a preferred layout (the BLFS layout patch) in 
> LFS/BLFS v8.2.
> 
> B/LFS-svn is shipping with v2.4.33 currently.
> 
> Alain (bug chaser for B/LFS and ALFS working toward editorship).

Looking at http://www.linuxfromscratch.org/blfs/view/svn/server/apache.html it 
doesn’t appear that you’re shipping httpd at all, instead you’re directing 
people to get httpd from the ASF, and are supplying a patch to make it work 
with LFS. Both of these activities are entirely fine.

As an aside - httpd has a —enable-layout option in configure that defines where 
things should go. If you patch the following file how you want it and submit it 
to us, we can formally support LFS out the box and you can remove the need for 
your patch:

https://svn.apache.org/repos/asf/httpd/sandbox/replacelimit/config.layout

Regards,
Graham
—



smime.p7s
Description: S/MIME cryptographic signature

Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)

[Stefan Eissing]

> Am 17.04.2018 um 19:18 schrieb William A Rowe Jr :
> 
>> On Tue, Apr 17, 2018 at 11:28 AM, Graham Leggett  wrote:
>> 
>> The distributions have been doing this nigh on two decades - the stability 
>> of a given software baseline which will not suddenly break at 3am some 
>> arbitrary Sunday in the middle of the holidays is the very product they’re 
>> selling. This works because they ship a baseline, plus carefully curated 
>> fixes as required by their communities, trading off the needs of their 
>> communities and stability.
> 
> So with respect to *our* communities...
> 
>> On Tue, Apr 17, 2018 at 11:17 AM, Graham Leggett  wrote:
>>> On 17 Apr 2018, at 6:08 PM, William A Rowe Jr  wrote:
>>> 
>>> No enhancement since 2011-12-19 has been presented for the collective
>>> community's scrutiny.
>> 
>> Again, I’m not following.
>> 
>> The architecture of v2.4 has been very stable, the need for breaking changes 
>> has been largely non existent, and the focus since 2011 has been to get 
>> changes backported to v2.4 instead.
>> 
>> To distill this down to raw numbers, there have been 1546 discrete backports 
>> (my simple grep of CHANGES) since 2011 - which has provided an enormous 
>> amount of enhancement for the collective community’s scrutiny.
> 
> And the corresponding number of regressions and behavior changes. None
> of these have enjoyed an "RC" or "beta", whatever one calls it, to
> validate before adoption - other than our claim of "best httpd yet".
> It has been an entirely new kitchen sink on every subversion release.

All my substantial functional additions had beta releases for months before 
being voted into the 2.4.x branch. With binary beta packages available for 
several platforms by several supporters.

William, this painting our world a dark and miserable place is coming back 
every few months. It is a disservice to the people who contribute changes here.

>> You seem to be making a mountain out of a molehill, I just don’t see the 
>> problem you’re trying to solve.
> 
> You are welcome to attribute this concern any way you like, and be
> satisfied with whatever yardstick you wish to measure it by. If you
> interpret our users as desiring enhancement and not stability, then
> those are the interests you should advocate. I'll leave this thread
> alone for another week again to give them the opportunity to chime in.

There are alternative ways to be creative and innovate than going through this 
PMC into a semi annual release.

Releasing a module (plus some small patches) on github opens ways for 
collaboration with people who like Apache and new stuff. Distros like Debian 
unstable and Fedora pick up stuff from there. PPAs for apt are made available. 
Steffen offers Windows builts.

The release cycle is hours, to the benefit of all interested. Be it a blocking 
bug fixed or a nice feature implemented. These are mostly people who do it for 
fun. Some even run large server clusters, so a „hobbyist“ label does not apply.

It‘s simply fun. It‘s how I think FOSS is supposed to work and has worked best 
in the past. I plan to continue doing it.

If that stuff makes it all back into the Apache svn is not that relevant to me. 
Because it‘s the least rewarding and fun part.

(I am just talking about my feelings here, YMMV.

Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)

[William A Rowe Jr]

On Tue, Apr 17, 2018 at 11:28 AM, Graham Leggett  wrote:
>
> The distributions have been doing this nigh on two decades - the stability of 
> a given software baseline which will not suddenly break at 3am some arbitrary 
> Sunday in the middle of the holidays is the very product they’re selling. 
> This works because they ship a baseline, plus carefully curated fixes as 
> required by their communities, trading off the needs of their communities and 
> stability.

So with respect to *our* communities...

On Tue, Apr 17, 2018 at 11:17 AM, Graham Leggett  wrote:
> On 17 Apr 2018, at 6:08 PM, William A Rowe Jr  wrote:
>
>> No enhancement since 2011-12-19 has been presented for the collective
>> community's scrutiny.
>
> Again, I’m not following.
>
> The architecture of v2.4 has been very stable, the need for breaking changes 
> has been largely non existent, and the focus since 2011 has been to get 
> changes backported to v2.4 instead.
>
> To distill this down to raw numbers, there have been 1546 discrete backports 
> (my simple grep of CHANGES) since 2011 - which has provided an enormous 
> amount of enhancement for the collective community’s scrutiny.

And the corresponding number of regressions and behavior changes. None
of these have enjoyed an "RC" or "beta", whatever one calls it, to
validate before adoption - other than our claim of "best httpd yet".
It has been an entirely new kitchen sink on every subversion release.

> You seem to be making a mountain out of a molehill, I just don’t see the 
> problem you’re trying to solve.

You are welcome to attribute this concern any way you like, and be
satisfied with whatever yardstick you wish to measure it by. If you
interpret our users as desiring enhancement and not stability, then
those are the interests you should advocate. I'll leave this thread
alone for another week again to give them the opportunity to chime in.

Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)

[Alain Toussaint]

> No
> distribution (that I am aware of) ships something called Apache httpd v2.4.29.

At LFS (linux from scratch), we're the exception confirming the rule of 
shipping v2.4.29 with the
single patch of defining a preferred layout (the BLFS layout patch) in LFS/BLFS 
v8.2.

B/LFS-svn is shipping with v2.4.33 currently.

Alain (bug chaser for B/LFS and ALFS working toward editorship).

Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)

[Graham Leggett]

On 17 Apr 2018, at 5:40 PM, William A Rowe Jr  wrote:

>> I’m not following the “all in vain”.
>> 
>> This patch in v2.4.33 was dine specifically to fix an issue in Xenial, and 
>> Ubuntu is on the case:
>> 
>> https://bugs.launchpad.net/ubuntu/+source/apache2/+bug/1750356
> 
> Then Ubuntu is distributing neither httpd 2.4.33 nor 2.4.29, as
> published by the Apache HTTP Project. This is another example of
> cherry picking a miscellany of fixes.

Yes. This is the very definition of the Ubuntu “Long Term Support” releases. It 
is also the very definition of “Redhat Enterprise Linux”.

> If a distributor shipped a source package of something called Apache
> httpd 2.4.29, which is obviously not .29 but .29+{stuff}, what would
> be our reaction?

No reaction.

There is no source of confusion. The distros all use (for example) v2.4.29 as 
their baseline version, and then a sub-version-number that to indicate their 
patch level on top of ours. No distribution (that I am aware of) ships 
something called Apache httpd v2.4.29.

The distributions have been doing this nigh on two decades - the stability of a 
given software baseline which will not suddenly break at 3am some arbitrary 
Sunday in the middle of the holidays is the very product they’re selling. This 
works because they ship a baseline, plus carefully curated fixes as required by 
their communities, trading off the needs of their communities and stability.

None of this is new.

It turns out that we, the httpd project (and apr), have had the exact same 
approach to stability that the distros have had for the last two decades. As a 
result, you can take an ASF supplied httpd RPM and drop it into Redhat 
Enterprise Linux and this “just works”, because our ABI guarantees align 
exactly with the ABI guarantees of the stable distros.

Regards,
Graham
—



smime.p7s
Description: S/MIME cryptographic signature

Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)

[Graham Leggett]

On 17 Apr 2018, at 6:08 PM, William A Rowe Jr  wrote:

> No enhancement since 2011-12-19 has been presented for the collective
> community's scrutiny.

Again, I’m not following.

The architecture of v2.4 has been very stable, the need for breaking changes 
has been largely non existent, and the focus since 2011 has been to get changes 
backported to v2.4 instead.

To distill this down to raw numbers, there have been 1546 discrete backports 
(my simple grep of CHANGES) since 2011 - which has provided an enormous amount 
of enhancement for the collective community’s scrutiny.

You seem to be making a mountain out of a molehill, I just don’t see the 
problem you’re trying to solve.

Regards,
Graham
—



smime.p7s
Description: S/MIME cryptographic signature

Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)

[William A Rowe Jr]

On Tue, Apr 17, 2018 at 10:50 AM, William A Rowe Jr  wrote:
>
> No enhancement since 2011-12-19 has been subjected to any community
> scrutiny. This was the date 2.3.16-beta for 2.4 was announced.

Sorry that statement is somewhat unfair...

* Anyone is welcome to "be a developer" and check out trunk, same for
2.4 branch. It simply isn't "published" till it is released.
* Anyone participating at dev@ can join in for three days of voting.
* PR watchers frequently test proposed fixes, some with new features.
* Steffen and others offer up binaries of proposed backports or
modules, e.g. the h2 and mod_md efforts.

The word "any" is way off-base. Trying this instead;

No enhancement since 2011-12-19 has been presented for the collective
community's scrutiny.

Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)

[William A Rowe Jr]

On Tue, Apr 17, 2018 at 9:47 AM, Graham Leggett  wrote:
> On 17 Apr 2018, at 4:41 PM, William A Rowe Jr  wrote:
>
>> We observe the "code freeze" effect (defined by three different
>> distributors) coupled with distributors deep distrust of our releases,
>> so by continuously polluting our version major.minor release with more
>> and more cruft, those users are denied not only the new cruft, but all
>> the bug fixes to the old cruft as well... there's really no other
>> explanation for the users of one of our most common distributions to
>> be locked out of several subversions worth of bugfix corrections.
>
> I’m lost - what problem are you trying to solve?

There is a second problem implied above, which I overlooked, sorry.

No enhancement since 2011-12-19 has been subjected to any community
scrutiny. This was the date 2.3.16-beta for 2.4 was announced.

Yes, patches go through test frameworks and peer review. But every
enhancement has been foisted on the user community without any
pre-adoption scrutiny.

This is made plain by the frequent number of rejected release
candidates, and by the equally frequent number of post-release
regression reports.

No enhancement I'm aware of has been rejected by the dev@ community;
eventually objections will be withdrawn, with enough committers will
rubber stamp whatever is in STATUS.

The project has been responsive to these regressions by releasing
fixes, which themselves are overloaded with new features and behavior
changes.

Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)

[Eric Covener]

> If a distributor shipped a source package of something called Apache
> httpd 2.4.29, which is obviously not .29 but .29+{stuff}, what would
> be our reaction?

The package name/filename/etc or the compiled-in server version?
For the former, it's already differentiated on most distros I've seen.
For the latter, I don't have any real concern as most people
understand if it's complex & packaged, it's patched.

-- 
Eric Covener
cove...@gmail.com

Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)

[William A Rowe Jr]

On Tue, Apr 17, 2018 at 9:47 AM, Graham Leggett  wrote:
> On 17 Apr 2018, at 4:41 PM, William A Rowe Jr  wrote:
>
>> And everything contributed to 2.4.33 release? All in vain. None of
>> that in this OS distribution, because, code freeze.
>
> I’m not following the “all in vain”.
>
> This patch in v2.4.33 was dine specifically to fix an issue in Xenial, and 
> Ubuntu is on the case:
>
> https://bugs.launchpad.net/ubuntu/+source/apache2/+bug/1750356

Then Ubuntu is distributing neither httpd 2.4.33 nor 2.4.29, as
published by the Apache HTTP Project. This is another example of
cherry picking a miscellany of fixes.

>> We observe the "code freeze" effect (defined by three different
>> distributors) coupled with distributors deep distrust of our releases,
>> so by continuously polluting our version major.minor release with more
>> and more cruft, those users are denied not only the new cruft, but all
>> the bug fixes to the old cruft as well... there's really no other
>> explanation for the users of one of our most common distributions to
>> be locked out of several subversions worth of bugfix corrections.
>
> I’m lost - what problem are you trying to solve?

The problem identified above, distributors falling into the role of
individually, project-by-project, release-by-release managing
versioning of what other modern software projects arbitrage in their
own subversion branches.

The use of the Apache HTTP Server mark itself is predicated on the
software shipped by Apache HTTP Project. So this forking leads to
interesting questions (probably permissible for combinations of code
released at different points by the project).

If a distributor shipped a source package of something called Apache
httpd 2.4.29, which is obviously not .29 but .29+{stuff}, what would
be our reaction?

Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)

[Graham Leggett]

On 15 Apr 2018, at 3:25 AM, Yehuda Katz  wrote:

> That also assumes the OS distributions pick up the point releases. RedHat 
> certainly doesn't pick up the new features, only bug fixes.

By design - that is what “Redhat Enterprise Linux” is.

Regards,
Graham
—



smime.p7s
Description: S/MIME cryptographic signature

Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)

[Graham Leggett]

On 17 Apr 2018, at 4:41 PM, William A Rowe Jr  wrote:

> And everything contributed to 2.4.33 release? All in vain. None of
> that in this OS distribution, because, code freeze.

I’m not following the “all in vain”.

This patch in v2.4.33 was dine specifically to fix an issue in Xenial, and 
Ubuntu is on the case:

https://bugs.launchpad.net/ubuntu/+source/apache2/+bug/1750356

> We observe the "code freeze" effect (defined by three different
> distributors) coupled with distributors deep distrust of our releases,
> so by continuously polluting our version major.minor release with more
> and more cruft, those users are denied not only the new cruft, but all
> the bug fixes to the old cruft as well... there's really no other
> explanation for the users of one of our most common distributions to
> be locked out of several subversions worth of bugfix corrections.

I’m lost - what problem are you trying to solve?

Regards,
Graham
—



smime.p7s
Description: S/MIME cryptographic signature

Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)

[William A Rowe Jr]

On Sat, Apr 14, 2018 at 8:48 AM, Jim Jagielski  wrote:
> IMO, the below ignores the impacts on OS distributors who
> provide httpd. We have seen how long it takes for them
> to go from 2.2 to 2.4...

They went to 2.4 once 2.4 was no longer beta. There is this concept
called "code freeze". At that point in the most modern OS distribution
(for the few weeks that lasts), Ubuntu 18.04 ships with...

Apache httpd 2.4.29
Apache apr 1.6.3
Apache apr-util 1.6.1
libcurl 7.58
OpenSSL 1.1.0g
nghttp2 1.30.0
brotli 1.0.3
Expat 2.2.5
jansson 2.11
libxml2 2.9.4
lua 5.3.3
PCRE 8.39 + 10.31
ZLib 1.2.11

How long will it take Ubuntu to pick up 2.4.next + OpenSSL 1.1.1 to
support TLSv1.3? Answer: next release, the 18.04 ship sailed.

And everything contributed to 2.4.33 release? All in vain. None of
that in this OS distribution, because, code freeze.

Nobody installing Ubuntu 18.04 finds TLSv1.3 from OpenSSL and their
consuming programs out of the box. This means 18.10, or 20.04, 2 years
from now - for those "stable" users.

The only thing this imaginary numbering question provokes is fear of
moving the project forwards. In the time its taken this project to
make minor tweaks around the edges in httpd, and jump forward by only
a handful of large leaps over 20 years, how many versions did our
primary open-source consumers release? Ubuntu 18.04 again - the only
thing almost as slow as httpd evolution has been lynx;

chromium 65  firefox 59  konqueror 17 lynx 2.8.9

Thanks David, and Nick, for trying to dispel this paranoia that
version numbers will cause users and distributors to flee the project.

Here's my concern, if .subversion meant bug fix (and bug fix, only)
then httpd distributed across Debian, Ubuntu, Redhat, Fedora,
Free/Open/NetBSD etc etc could correspond to something the httpd
project released. Because they all cherry pick only "necessary" bug
fixes (and each define those differently), not one of them distributes
*Apache Software Foundation Apache Web Server httpd*. By mashing all
the fun new stuff into the same numbers because adoption yadda yadda,
none of them can turn to httpd for the necessary fixes for their
distribution, and they certainly can't simply review and rubber stamp
our subversion release for the "right" set of bug fixes.

The irony in all this is that I was taught "version numbers are cheap"
fairly early, by this very project. No truth-in-advertising, that
"subversion numbers are cheap, major version numbers are a heavy lift
of 20 years of baggage."

You also claimed some delay in 2.4 adoption, when there was none in
fact. In 2014;

January; OpenSUSE 13 ships httpd 2.4.10
March; Ubuntu 14.04 ships httpd 2.4.7
April; RedHat 7 ships httpd 2.4.6

We observe the "code freeze" effect (defined by three different
distributors) coupled with distributors deep distrust of our releases,
so by continuously polluting our version major.minor release with more
and more cruft, those users are denied not only the new cruft, but all
the bug fixes to the old cruft as well... there's really no other
explanation for the users of one of our most common distributions to
be locked out of several subversions worth of bugfix corrections.

Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)

[David Zuelke]

On Sat, Apr 14, 2018 at 4:34 PM, Nick Kew  wrote:
>
>> On 14 Apr 2018, at 14:48, Jim Jagielski  wrote:
>>
>> IMO, the below ignores the impacts on OS distributors who
>> provide httpd. We have seen how long it takes for them
>> to go from 2.2 to 2.4... I can't imagine the impact for our
>> end user community if "new features" cause a minor
>> bump all the time and we "force" distributions for
>> 2.4->2.6->2.8->2.10…
>
> Chicken  httpd version numbers creep in a petty pace from year to year,
> and packagers do likewise.  Contrast a product like, say, Firefox, where major
> versions just whoosh by, and distros increment theirs every few months.
>

It's not like distros pick up patch releases anyway. They backport
fixes to whatever they "froze" to upon first release, and that's it.

Debian and Ubuntu, for instance, just pick the latest PHP that's
released at the time the freeze for a version happens, and that's it.


>>> On Apr 13, 2018, at 2:28 PM, David Zuelke  wrote:
>>>
>>> Remember the thread I started on that quite a while ago? ;)
>
> Nope.

https://lists.apache.org/thread.html/9afe84b5c2e7691f0190210e2377a6d504a6a77ff1481812f44f65d4@%3Cdev.httpd.apache.org%3E

>>> - x.y.0 for new features
>>> - x.y.z for bugfixes only
>>> - stop the endless backporting
>>> - make x.y.0 releases more often
>
> An issue there is the API/ABI promise.  We are a stable product, and one of 
> our
> virtues is the guarantee that a third-party module written for x.y.z will 
> continue to
> work at both source and binary level for x.y.(z+n).
>
> Frequent x.y.0 releases devalue that promise unless we extend it to x.(y+m).*,
> which would in turn push us into new x.0.0 releases, and raise new questions
> over the whole organisation of our repos.
>
> I’m not saying you’re wrong: in fact I think there’s merit in the proposal.
> But it would need a considered roadmap from here to there.

Well, one important thing to keep in mind is that a x.y.0 release
doesn't preclude the x.(y-1) series from receiving fixes. Users don't
have to update immediately if they're using third-party modules, as
there would still be bug fixes for a while, and eventually only
security fixes, before the x.(y-1) series would be fully EOL.

PHP has a very predictable timeline for this:
http://php.net/supported-versions.php

It's also worth noting that with more frequent x.y.0 releases (say,
one per year), it's likely that internal changes will be a lot smaller
and more incremental. PHP is in a similar situation to HTTPD with its
extensions system, and extensions that worked with PHP 7.0 either
compiled fine against 7.1 and later 7.2 out of the box, or required
only very few modifications.

David

Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)

[Ruediger Pluem]

On 04/15/2018 03:25 AM, Yehuda Katz wrote:
> On Sat, Apr 14, 2018 at 9:48 AM, Jim Jagielski  > wrote:
> 
> IMO, the below ignores the impacts on OS distributors who
> provide httpd. We have seen how long it takes for them
> to go from 2.2 to 2.4... I can't imagine the impact for our
> end user community if "new features" cause a minor
> bump all the time and we "force" distributions for
> 2.4->2.6->2.8->2.10...
> 
> Just my 2c
> 
> 
> That also assumes the OS distributions pick up the point releases. RedHat 
> certainly doesn't pick up the new features,
> only bug fixes.

But in this case users of those binaries shouldn't be affected by the 
regressions as they only receive bug fix backports
:-P.
Seriously, we also had regressions just in bug fixes and the feature 
backporting makes it sometimes harder to correctly
backport pure bug fixes for these distributions as the code changes are bigger 
in the stable branch because of the
feature backports. Of course this is not our direct issue and main concern here.
One idea that comes to my mind is whether we should have an "LTS" version that 
only receives bugfixes and another
"stable" branch that receives new feature (on the same level of API 
compatibility we currently grant for our stable
branches). E.g. you could go with a major release X.Y.0.Z and after some minor 
releases Z which still allow
feature and bugfixes to be backported in an API compatible way (to give the new 
major some time to stabilize) split of
X.Y.1.Z to allow feature backports in an API compatible way and allow only bug 
fix backports to X.Y.0.Z.
Questions that pop up with this are:

1. Is there enough manpower and willingness to maintain this?
2. How will the commercial 3rd parties handle this? Will they only support 
X.Y.0.Z or will they also support X.Y.1.Z?
   From an API point of view it doesn't matter as X.Y.0 and X.Y.1 follow the 
same API guarantee. X.Y.1 just has a
   bigger regression risk.


Other typical suspects are:

1. Improve testing suite(s).
2. Give the future release a broader real life testing exposure. The question 
is how we can get to this. I am not sure
   if RC release will help here, because it requires a sufficient amount of 
people to use and test them. Seeing RC on a
   release might make them saying: Nah, let others do that testing I will wait 
until RC is gone and take it then.
   So I am not sure if RC releases will improve the exposure compared to the 
current exposure during the voting.
   If it is just a matter of time (voting usually "only" takes 72 hours, 
compared to a RC release which would be around
   longer before the next RC or final release) it could help indeed.

Sorry for the rant.


Regards

Rüdiger

Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)

[Ruediger Pluem]

On 04/14/2018 04:34 PM, Nick Kew wrote:
> 
>> On 14 Apr 2018, at 14:48, Jim Jagielski  wrote:
>>
>> IMO, the below ignores the impacts on OS distributors who
>> provide httpd. We have seen how long it takes for them
>> to go from 2.2 to 2.4... I can't imagine the impact for our
>> end user community if "new features" cause a minor
>> bump all the time and we "force" distributions for
>> 2.4->2.6->2.8->2.10…
> 
> Chicken  httpd version numbers creep in a petty pace from year to year,
> and packagers do likewise.  Contrast a product like, say, Firefox, where major
> versions just whoosh by, and distros increment theirs every few months.
> 
>> Just my 2c
> 
> Indeed, a change needs to be a considered thing, and there are issues.
> 
>>> On Apr 13, 2018, at 2:28 PM, David Zuelke  wrote:
>>>
>>> Remember the thread I started on that quite a while ago? ;)
> 
> Nope.
> 
>>> - x.y.0 for new features
>>> - x.y.z for bugfixes only
>>> - stop the endless backporting
>>> - make x.y.0 releases more often
> 
> An issue there is the API/ABI promise.  We are a stable product, and one of 
> our
> virtues is the guarantee that a third-party module written for x.y.z will 
> continue to
> work at both source and binary level for x.y.(z+n).

This is the biggest issue here and where we need some way to keep this promise 
for a longer time. Especially commercial
module suppliers are extremely slow there, even if a recompile of their source 
against the new version would already do
like it did in many cases during the 2.2 -> 2.4 transition.
I am not sure currently how our user base looks like:

1. How many take it from OS distros?
2. How many take it from sources that deliver the latest 2.4 or compile 
themselves.
3. How many use closed source 3rd party modules that need long term API/ABI 
promises?

Regards

Rüdiger

Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)

[Yehuda Katz]

On Sat, Apr 14, 2018 at 9:48 AM, Jim Jagielski  wrote:

> IMO, the below ignores the impacts on OS distributors who
> provide httpd. We have seen how long it takes for them
> to go from 2.2 to 2.4... I can't imagine the impact for our
> end user community if "new features" cause a minor
> bump all the time and we "force" distributions for
> 2.4->2.6->2.8->2.10...
>
> Just my 2c
>
>
That also assumes the OS distributions pick up the point releases. RedHat
certainly doesn't pick up the new features, only bug fixes.

- Y



> > On Apr 13, 2018, at 2:28 PM, David Zuelke 
> wrote:
> >
> > Remember the thread I started on that quite a while ago? ;)
> >
> > IMO:
> >
> > - x.y.0 for new features
> > - x.y.z for bugfixes only
> > - stop the endless backporting
> > - make x.y.0 releases more often
> > - x.y.0 goes through alpha, beta, RC phases
> > - x.y.z goes through RC phases
> >
> > That's how PHP has been doing it for a few years, and it's amazing how
> > well it works, how few regressions there are, and how predictable the
> > cycle is (they cut an x.y.zRC1 every four weeks like clockwork, with
> > exceptions only around late December because of holiday season).
> >
> > This would also fix all the confusing cases where two or three faulty
> > releases get made, end up in the changelog, but ultimately are never
> > released.
> >
> >
> > On Fri, Apr 13, 2018 at 5:28 PM, William A Rowe Jr 
> wrote:
> >> Terrific analysis! But on the meta-question...
> >>
> >> Instead of changing the behavior of httpd on each and every subversion
> bump,
> >> is it time to revisit our revisioning discipline and hygiene?
> >>
> >> I promise to stay out of such discussion provided that one equally
> stubborn
> >> and intractable PMC member agrees to do the same, and let the balance
> of the
> >> PMC make our decision, moving forwards.
> >>
> >> On Fri, Apr 13, 2018, 06:11 Joe Orton  wrote:
> >>>
> >>> On Thu, Apr 12, 2018 at 09:38:46PM +0200, Ruediger Pluem wrote:
>  On 04/12/2018 09:28 AM, Joe Orton wrote:
> > But logged is:
> >
> > ::1 - - [12/Apr/2018:08:11:12 +0100] "GET /agag HTTP/1.1" 404 12
> > HTTPS=on SNI=localhost.localdomain
> > 127.0.0.1 - - [12/Apr/2018:08:11:15 +0100] "GET /agag HTTP/1.1" 404
> 12
> > HTTPS=- SNI=-
> >
> > Now mod_ssl only sees the "off" SSLSrvConfigRec in the second vhost
> so
> > the logging is wrong.
> 
>  What does the same test result in with 2.4.29?
> >>>
> >>> Excellent question, I should have checked that.  Long e-mail follows,
> >>> sorry.
> >>>
> >>> In fact it is the same with 2.4.29, because the SSLSrvConfigRec
> >>> associated with the vhost's server_rec is the same as the default/base
> >>> (non-SSL) server_rec, aka base_server passed to post_config hooks aka
> >>> the ap_server_conf global.
> >>>
> >>> So, maybe I understand this a bit better now.
> >>>
> >>> Config with three vhosts / server_rec structs:
> >>> a) base server config :80 non-SSL (<-- ap_server_conf/base_server)
> >>> b) alpha vhost :443, explicit SSLEngine on, SSLCertificateFile etc
> >>> c) beta vhost :443, no SSL*
> >>>
> >>> For 2.4.29 mod_ssl config derived is:
> >>> a) SSLSrvConfigRec for base_server = { whatever config at global scope
> }
> >>> b) SSLSrvConfigRec for alpha = { sc->enabled = TRUE, ... }
> >>> c) SSLSrvConfigRec pointer for beta == SSLSrvConfigRec for base_server
> >>>   in the lookup vector (pointer is copied prior to ALWAYS_MERGE flag)
> >>>
> >>> For 2.4.33 it is:
> >>> a) and b) exactly as before
> >>> c) separate SSLSrvConfigRec for beta = { merged copy of config at
> global }
> >>>   time because of the ALWAYS_MERGE flag, i.e. still sc->enabled = UNSET
> >>>
> >>> When running ssl_init_Module(post_config hook), with 2.4.29:
> >>> - SSLSrvConfig(base_server)->enabled = FALSE because UNSET previously
> >>> - SSLSrvConfig(base_server)->vhost_id gets overwritten with vhost_id
> >>>  for beta vhost because it's later in the loop and there's no check
> >>>
> >>> And with 2.4.33:
> >>> - SSLSrvConfig(beta)->enabled is UNSET but gets flipped to ENABLED,
> >>>  then startup fails (the issue in question)
> >>>
> >>> w/my patch for 2.4.33:
> >>> - SSLSrvConfig(beta)->enabled is FALSE and startup works
> >>>
> >>> At run-time a request via SSL which matches the beta vhost via
> SNI/Host:
> >>>
> >>> For 2.4.29:
> >>> - r->server is the beta vhost and mySrvConfig(r->server) still gives
> >>>  you the ***base_server*** SSLSrvConfigRec i.e. sc->enabled=FALSE
> >>> - thus e.g. ssl_hook_Fixup() does nada
> >>>
> >>> For 2.4.33 plus my patch:
> >>> - r->server is the beta vhost and mySrvConfig(r->server) gives
> >>>  you the SSLSrvConfigRec which is also sc->enabled = FALSE
> >>> - thus e.g. ssl_hook_Fixup() also does nada
> >>>
> >>> I was trying to convince myself whether mySrvConfig(r->server) is going
> >>> to change between 2.4.29 and .33+patch in this case, and I think it
> >>> 

Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)

[Nick Kew]

> On 14 Apr 2018, at 14:48, Jim Jagielski  wrote:
> 
> IMO, the below ignores the impacts on OS distributors who
> provide httpd. We have seen how long it takes for them
> to go from 2.2 to 2.4... I can't imagine the impact for our
> end user community if "new features" cause a minor
> bump all the time and we "force" distributions for
> 2.4->2.6->2.8->2.10…

Chicken  httpd version numbers creep in a petty pace from year to year,
and packagers do likewise.  Contrast a product like, say, Firefox, where major
versions just whoosh by, and distros increment theirs every few months.

> Just my 2c

Indeed, a change needs to be a considered thing, and there are issues.

>> On Apr 13, 2018, at 2:28 PM, David Zuelke  wrote:
>> 
>> Remember the thread I started on that quite a while ago? ;)

Nope.

>> - x.y.0 for new features
>> - x.y.z for bugfixes only
>> - stop the endless backporting
>> - make x.y.0 releases more often

An issue there is the API/ABI promise.  We are a stable product, and one of our
virtues is the guarantee that a third-party module written for x.y.z will 
continue to
work at both source and binary level for x.y.(z+n).

Frequent x.y.0 releases devalue that promise unless we extend it to x.(y+m).*,
which would in turn push us into new x.0.0 releases, and raise new questions
over the whole organisation of our repos.

I’m not saying you’re wrong: in fact I think there’s merit in the proposal.
But it would need a considered roadmap from here to there.

— 
Nick Kew

Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)

[Jim Jagielski]

IMO, the below ignores the impacts on OS distributors who
provide httpd. We have seen how long it takes for them
to go from 2.2 to 2.4... I can't imagine the impact for our
end user community if "new features" cause a minor
bump all the time and we "force" distributions for
2.4->2.6->2.8->2.10...

Just my 2c

> On Apr 13, 2018, at 2:28 PM, David Zuelke  wrote:
> 
> Remember the thread I started on that quite a while ago? ;)
> 
> IMO:
> 
> - x.y.0 for new features
> - x.y.z for bugfixes only
> - stop the endless backporting
> - make x.y.0 releases more often
> - x.y.0 goes through alpha, beta, RC phases
> - x.y.z goes through RC phases
> 
> That's how PHP has been doing it for a few years, and it's amazing how
> well it works, how few regressions there are, and how predictable the
> cycle is (they cut an x.y.zRC1 every four weeks like clockwork, with
> exceptions only around late December because of holiday season).
> 
> This would also fix all the confusing cases where two or three faulty
> releases get made, end up in the changelog, but ultimately are never
> released.
> 
> 
> On Fri, Apr 13, 2018 at 5:28 PM, William A Rowe Jr  
> wrote:
>> Terrific analysis! But on the meta-question...
>> 
>> Instead of changing the behavior of httpd on each and every subversion bump,
>> is it time to revisit our revisioning discipline and hygiene?
>> 
>> I promise to stay out of such discussion provided that one equally stubborn
>> and intractable PMC member agrees to do the same, and let the balance of the
>> PMC make our decision, moving forwards.
>> 
>> On Fri, Apr 13, 2018, 06:11 Joe Orton  wrote:
>>> 
>>> On Thu, Apr 12, 2018 at 09:38:46PM +0200, Ruediger Pluem wrote:
 On 04/12/2018 09:28 AM, Joe Orton wrote:
> But logged is:
> 
> ::1 - - [12/Apr/2018:08:11:12 +0100] "GET /agag HTTP/1.1" 404 12
> HTTPS=on SNI=localhost.localdomain
> 127.0.0.1 - - [12/Apr/2018:08:11:15 +0100] "GET /agag HTTP/1.1" 404 12
> HTTPS=- SNI=-
> 
> Now mod_ssl only sees the "off" SSLSrvConfigRec in the second vhost so
> the logging is wrong.
 
 What does the same test result in with 2.4.29?
>>> 
>>> Excellent question, I should have checked that.  Long e-mail follows,
>>> sorry.
>>> 
>>> In fact it is the same with 2.4.29, because the SSLSrvConfigRec
>>> associated with the vhost's server_rec is the same as the default/base
>>> (non-SSL) server_rec, aka base_server passed to post_config hooks aka
>>> the ap_server_conf global.
>>> 
>>> So, maybe I understand this a bit better now.
>>> 
>>> Config with three vhosts / server_rec structs:
>>> a) base server config :80 non-SSL (<-- ap_server_conf/base_server)
>>> b) alpha vhost :443, explicit SSLEngine on, SSLCertificateFile etc
>>> c) beta vhost :443, no SSL*
>>> 
>>> For 2.4.29 mod_ssl config derived is:
>>> a) SSLSrvConfigRec for base_server = { whatever config at global scope }
>>> b) SSLSrvConfigRec for alpha = { sc->enabled = TRUE, ... }
>>> c) SSLSrvConfigRec pointer for beta == SSLSrvConfigRec for base_server
>>>   in the lookup vector (pointer is copied prior to ALWAYS_MERGE flag)
>>> 
>>> For 2.4.33 it is:
>>> a) and b) exactly as before
>>> c) separate SSLSrvConfigRec for beta = { merged copy of config at global }
>>>   time because of the ALWAYS_MERGE flag, i.e. still sc->enabled = UNSET
>>> 
>>> When running ssl_init_Module(post_config hook), with 2.4.29:
>>> - SSLSrvConfig(base_server)->enabled = FALSE because UNSET previously
>>> - SSLSrvConfig(base_server)->vhost_id gets overwritten with vhost_id
>>>  for beta vhost because it's later in the loop and there's no check
>>> 
>>> And with 2.4.33:
>>> - SSLSrvConfig(beta)->enabled is UNSET but gets flipped to ENABLED,
>>>  then startup fails (the issue in question)
>>> 
>>> w/my patch for 2.4.33:
>>> - SSLSrvConfig(beta)->enabled is FALSE and startup works
>>> 
>>> At run-time a request via SSL which matches the beta vhost via SNI/Host:
>>> 
>>> For 2.4.29:
>>> - r->server is the beta vhost and mySrvConfig(r->server) still gives
>>>  you the ***base_server*** SSLSrvConfigRec i.e. sc->enabled=FALSE
>>> - thus e.g. ssl_hook_Fixup() does nada
>>> 
>>> For 2.4.33 plus my patch:
>>> - r->server is the beta vhost and mySrvConfig(r->server) gives
>>>  you the SSLSrvConfigRec which is also sc->enabled = FALSE
>>> - thus e.g. ssl_hook_Fixup() also does nada
>>> 
>>> I was trying to convince myself whether mySrvConfig(r->server) is going
>>> to change between 2.4.29 and .33+patch in this case, and I think it
>>> should be identical, because it is *only* the handling of ->enabled
>>> which has changed with _ALWAYS_MERGE.
>>> 
>>> TL;DR:
>>> 1. my head hurts
>>> 2. I think my patch is OK
>>> 
>>> Anyone read this far?

Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)

[David Zuelke]

Remember the thread I started on that quite a while ago? ;)

IMO:

- x.y.0 for new features
- x.y.z for bugfixes only
- stop the endless backporting
- make x.y.0 releases more often
- x.y.0 goes through alpha, beta, RC phases
- x.y.z goes through RC phases

That's how PHP has been doing it for a few years, and it's amazing how
well it works, how few regressions there are, and how predictable the
cycle is (they cut an x.y.zRC1 every four weeks like clockwork, with
exceptions only around late December because of holiday season).

This would also fix all the confusing cases where two or three faulty
releases get made, end up in the changelog, but ultimately are never
released.


On Fri, Apr 13, 2018 at 5:28 PM, William A Rowe Jr  wrote:
> Terrific analysis! But on the meta-question...
>
> Instead of changing the behavior of httpd on each and every subversion bump,
> is it time to revisit our revisioning discipline and hygiene?
>
> I promise to stay out of such discussion provided that one equally stubborn
> and intractable PMC member agrees to do the same, and let the balance of the
> PMC make our decision, moving forwards.
>
> On Fri, Apr 13, 2018, 06:11 Joe Orton  wrote:
>>
>> On Thu, Apr 12, 2018 at 09:38:46PM +0200, Ruediger Pluem wrote:
>> > On 04/12/2018 09:28 AM, Joe Orton wrote:
>> > > But logged is:
>> > >
>> > > ::1 - - [12/Apr/2018:08:11:12 +0100] "GET /agag HTTP/1.1" 404 12
>> > > HTTPS=on SNI=localhost.localdomain
>> > > 127.0.0.1 - - [12/Apr/2018:08:11:15 +0100] "GET /agag HTTP/1.1" 404 12
>> > > HTTPS=- SNI=-
>> > >
>> > > Now mod_ssl only sees the "off" SSLSrvConfigRec in the second vhost so
>> > > the logging is wrong.
>> >
>> > What does the same test result in with 2.4.29?
>>
>> Excellent question, I should have checked that.  Long e-mail follows,
>> sorry.
>>
>> In fact it is the same with 2.4.29, because the SSLSrvConfigRec
>> associated with the vhost's server_rec is the same as the default/base
>> (non-SSL) server_rec, aka base_server passed to post_config hooks aka
>> the ap_server_conf global.
>>
>> So, maybe I understand this a bit better now.
>>
>> Config with three vhosts / server_rec structs:
>> a) base server config :80 non-SSL (<-- ap_server_conf/base_server)
>> b) alpha vhost :443, explicit SSLEngine on, SSLCertificateFile etc
>> c) beta vhost :443, no SSL*
>>
>> For 2.4.29 mod_ssl config derived is:
>> a) SSLSrvConfigRec for base_server = { whatever config at global scope }
>> b) SSLSrvConfigRec for alpha = { sc->enabled = TRUE, ... }
>> c) SSLSrvConfigRec pointer for beta == SSLSrvConfigRec for base_server
>>in the lookup vector (pointer is copied prior to ALWAYS_MERGE flag)
>>
>> For 2.4.33 it is:
>> a) and b) exactly as before
>> c) separate SSLSrvConfigRec for beta = { merged copy of config at global }
>>time because of the ALWAYS_MERGE flag, i.e. still sc->enabled = UNSET
>>
>> When running ssl_init_Module(post_config hook), with 2.4.29:
>> - SSLSrvConfig(base_server)->enabled = FALSE because UNSET previously
>> - SSLSrvConfig(base_server)->vhost_id gets overwritten with vhost_id
>>   for beta vhost because it's later in the loop and there's no check
>>
>> And with 2.4.33:
>> - SSLSrvConfig(beta)->enabled is UNSET but gets flipped to ENABLED,
>>   then startup fails (the issue in question)
>>
>> w/my patch for 2.4.33:
>> - SSLSrvConfig(beta)->enabled is FALSE and startup works
>>
>> At run-time a request via SSL which matches the beta vhost via SNI/Host:
>>
>> For 2.4.29:
>> - r->server is the beta vhost and mySrvConfig(r->server) still gives
>>   you the ***base_server*** SSLSrvConfigRec i.e. sc->enabled=FALSE
>> - thus e.g. ssl_hook_Fixup() does nada
>>
>> For 2.4.33 plus my patch:
>> - r->server is the beta vhost and mySrvConfig(r->server) gives
>>   you the SSLSrvConfigRec which is also sc->enabled = FALSE
>> - thus e.g. ssl_hook_Fixup() also does nada
>>
>> I was trying to convince myself whether mySrvConfig(r->server) is going
>> to change between 2.4.29 and .33+patch in this case, and I think it
>> should be identical, because it is *only* the handling of ->enabled
>> which has changed with _ALWAYS_MERGE.
>>
>> TL;DR:
>> 1. my head hurts
>> 2. I think my patch is OK
>>
>> Anyone read this far?

Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)

[William A Rowe Jr]

Terrific analysis! But on the meta-question...

Instead of changing the behavior of httpd on each and every subversion
bump, is it time to revisit our revisioning discipline and hygiene?

I promise to stay out of such discussion provided that one equally stubborn
and intractable PMC member agrees to do the same, and let the balance of
the PMC make our decision, moving forwards.

On Fri, Apr 13, 2018, 06:11 Joe Orton  wrote:

> On Thu, Apr 12, 2018 at 09:38:46PM +0200, Ruediger Pluem wrote:
> > On 04/12/2018 09:28 AM, Joe Orton wrote:
> > > But logged is:
> > >
> > > ::1 - - [12/Apr/2018:08:11:12 +0100] "GET /agag HTTP/1.1" 404 12
> HTTPS=on SNI=localhost.localdomain
> > > 127.0.0.1 - - [12/Apr/2018:08:11:15 +0100] "GET /agag HTTP/1.1" 404 12
> HTTPS=- SNI=-
> > >
> > > Now mod_ssl only sees the "off" SSLSrvConfigRec in the second vhost so
> > > the logging is wrong.
> >
> > What does the same test result in with 2.4.29?
>
> Excellent question, I should have checked that.  Long e-mail follows,
> sorry.
>
> In fact it is the same with 2.4.29, because the SSLSrvConfigRec
> associated with the vhost's server_rec is the same as the default/base
> (non-SSL) server_rec, aka base_server passed to post_config hooks aka
> the ap_server_conf global.
>
> So, maybe I understand this a bit better now.
>
> Config with three vhosts / server_rec structs:
> a) base server config :80 non-SSL (<-- ap_server_conf/base_server)
> b) alpha vhost :443, explicit SSLEngine on, SSLCertificateFile etc
> c) beta vhost :443, no SSL*
>
> For 2.4.29 mod_ssl config derived is:
> a) SSLSrvConfigRec for base_server = { whatever config at global scope }
> b) SSLSrvConfigRec for alpha = { sc->enabled = TRUE, ... }
> c) SSLSrvConfigRec pointer for beta == SSLSrvConfigRec for base_server
>in the lookup vector (pointer is copied prior to ALWAYS_MERGE flag)
>
> For 2.4.33 it is:
> a) and b) exactly as before
> c) separate SSLSrvConfigRec for beta = { merged copy of config at global }
>time because of the ALWAYS_MERGE flag, i.e. still sc->enabled = UNSET
>
> When running ssl_init_Module(post_config hook), with 2.4.29:
> - SSLSrvConfig(base_server)->enabled = FALSE because UNSET previously
> - SSLSrvConfig(base_server)->vhost_id gets overwritten with vhost_id
>   for beta vhost because it's later in the loop and there's no check
>
> And with 2.4.33:
> - SSLSrvConfig(beta)->enabled is UNSET but gets flipped to ENABLED,
>   then startup fails (the issue in question)
>
> w/my patch for 2.4.33:
> - SSLSrvConfig(beta)->enabled is FALSE and startup works
>
> At run-time a request via SSL which matches the beta vhost via SNI/Host:
>
> For 2.4.29:
> - r->server is the beta vhost and mySrvConfig(r->server) still gives
>   you the ***base_server*** SSLSrvConfigRec i.e. sc->enabled=FALSE
> - thus e.g. ssl_hook_Fixup() does nada
>
> For 2.4.33 plus my patch:
> - r->server is the beta vhost and mySrvConfig(r->server) gives
>   you the SSLSrvConfigRec which is also sc->enabled = FALSE
> - thus e.g. ssl_hook_Fixup() also does nada
>
> I was trying to convince myself whether mySrvConfig(r->server) is going
> to change between 2.4.29 and .33+patch in this case, and I think it
> should be identical, because it is *only* the handling of ->enabled
> which has changed with _ALWAYS_MERGE.
>
> TL;DR:
> 1. my head hurts
> 2. I think my patch is OK
>
> Anyone read this far?
>