Re: [VOTE] Release httpd-2.4.30

[Jim Jagielski]

I would suggest using the scripts from

https://svn.apache.org/repos/asf/httpd/site/trunk/tools 


for as much of the work as possible since they have been used
and vetted for several years.

> On Feb 19, 2018, at 7:28 PM, drugg...@primary.net wrote:
> 
> Ah, I see. I created a script that does the tagging based on the directions 
> here.
> http://httpd.apache.org/dev/release.html
> 
> It was unclear in those instructions that one should commit the change to 
> AP_SERVER_DEVBUILD_BOOLEAN.
> 
> Instead I did the following:
> ...
> #Set AP_SERVER_DEVBUILD_BOOLEAN to 0 in include/ap_release.h.
> perl -pi -e 's/(#define\s+AP_SERVER_DEVBUILD_BOOLEAN\s+)\d/${1}0/g' 
> include/ap_release.h
> 
> #Create an official X.Y.Z tag based on the candidate tree.
> svn copy "$src_dir" "$tags_dir/$version"
> 
> #Revert the change to include/ap_release.h setting 
> AP_SERVER_DEVBUILD_BOOLEAN back to 1, and bump AP_SERVER_PATCHLEVEL_NUMBER
> perl -pi -e '
>  s/(#define\s+AP_SERVER_DEVBUILD_BOOLEAN\s+)\d/${1}1/g;
> 
>  if(/(#define\s+AP_SERVER_PATCHLEVEL_NUMBER\s+)(\d+)$/){
>$new = $2 + 1;
>$_ = "${1}${new}\n";
>  }
>  ' include/ap_release.h
> ...
> 
> This begets a tarball that has the Boolean set to 0, but no 
> commit/revert/bump (instead just an apparent bump):
> http://svn.apache.org/viewvc/httpd/httpd/tags/2.4.30/include/ap_release.h?view=markup#l47
> 
> It makes sense that the tag comes from a specific commit where the variable 
> was flipped...  Should I adjust the script and retry?
> 
> -- 
> Daniel Ruggeri
> 
> From: Jim Jagielski [mailto:j...@jagunet.com] 
> Sent: Monday, February 19, 2018 9:46 AM
> To: dev@httpd.apache.org
> Subject: Re: [VOTE] Release httpd-2.4.30
> 
> Hmmm... I'm not seeing the patch where AP_SERVER_DEVBUILD_BOOLEAN
> in ap_release.h is set to 0
> 
> How does your release process work? What we've always
> done is make the req changes to the branch and then copy
> from that branch to the tag. So the tag itself must refer to
> a specific SVN number on the http-2.4 branch but I'm
> not seeing where that is done.
> 
> 
> On Feb 19, 2018, at 9:54 AM, mailto:drugg...@primary.net wrote:
> 
> Hi, all;
>   Please find below the proposed release tarball and signatures:
> https://dist.apache.org/repos/dist/dev/httpd/
> 
> I would like to call a VOTE over the next few days to release this candidate 
> tarball as 2.4.30:
> 
> [ ] +1: It’s not just good, it’s good enough!
> [ ] +0: Let’s have a talk…
> [ ] -1: There’s trouble in paradise. Here’s what’s wrong.
> -- 
> Daniel Ruggeri
> 
> 

RE: [VOTE] Release httpd-2.4.30

[DRuggeri]

Ah, I see. I created a script that does the tagging based on the directions 
here.
http://httpd.apache.org/dev/release.html

It was unclear in those instructions that one should commit the change to 
AP_SERVER_DEVBUILD_BOOLEAN.

Instead I did the following:
...
#Set AP_SERVER_DEVBUILD_BOOLEAN to 0 in include/ap_release.h.
perl -pi -e 's/(#define\s+AP_SERVER_DEVBUILD_BOOLEAN\s+)\d/${1}0/g' 
include/ap_release.h

#Create an official X.Y.Z tag based on the candidate tree.
svn copy "$src_dir" "$tags_dir/$version"

#Revert the change to include/ap_release.h setting 
AP_SERVER_DEVBUILD_BOOLEAN back to 1, and bump AP_SERVER_PATCHLEVEL_NUMBER
perl -pi -e '
  s/(#define\s+AP_SERVER_DEVBUILD_BOOLEAN\s+)\d/${1}1/g;

  if(/(#define\s+AP_SERVER_PATCHLEVEL_NUMBER\s+)(\d+)$/){
$new = $2 + 1;
$_ = "${1}${new}\n";
  }
  ' include/ap_release.h
...

This begets a tarball that has the Boolean set to 0, but no commit/revert/bump 
(instead just an apparent bump):
http://svn.apache.org/viewvc/httpd/httpd/tags/2.4.30/include/ap_release.h?view=markup#l47

It makes sense that the tag comes from a specific commit where the variable was 
flipped...  Should I adjust the script and retry?

-- 
Daniel Ruggeri

From: Jim Jagielski [mailto:j...@jagunet.com] 
Sent: Monday, February 19, 2018 9:46 AM
To: dev@httpd.apache.org
Subject: Re: [VOTE] Release httpd-2.4.30

Hmmm... I'm not seeing the patch where AP_SERVER_DEVBUILD_BOOLEAN
in ap_release.h is set to 0

How does your release process work? What we've always
done is make the req changes to the branch and then copy
from that branch to the tag. So the tag itself must refer to
a specific SVN number on the http-2.4 branch but I'm
not seeing where that is done.


On Feb 19, 2018, at 9:54 AM, mailto:drugg...@primary.net wrote:

Hi, all;
   Please find below the proposed release tarball and signatures:
https://dist.apache.org/repos/dist/dev/httpd/
 
I would like to call a VOTE over the next few days to release this candidate 
tarball as 2.4.30:
 
[ ] +1: It’s not just good, it’s good enough!
[ ] +0: Let’s have a talk…
[ ] -1: There’s trouble in paradise. Here’s what’s wrong.
-- 
Daniel Ruggeri

Re: win32: disable shared LDAP cache by default

[Eric Covener]

I am hoping this is fixed by
http://svn.apache.org/viewvc?view=revision=182481 which I
stumbled onto from another direction.

On Thu, Aug 3, 2017 at 5:58 AM, Stefan Sperling  wrote:
> There are numerous reports of Apache HTTPD looping forever on Windows
> unless the LDAPSharedCacheSize option is set to zero.
>
> See for instance:
> https://svn.haxx.se/users/archive-2014-05/.shtml
> https://subversion.open.collab.net/ds/viewMessage.do?dsMessageId=564176=3
> https://subversion.open.collab.net/ds/viewMessage.do?dsForumId=3=browseAll=539507
> https://stackoverflow.com/questions/44542654/collabnet-subversion-server-reaching-cpu-100-because-of-httpd-exe-process
>
> I looked around for a while but don't know yet if a corresponding issue
> in the HTTPD bug tracker exists. Does anyone know?
>
> On the surface it looks like a memory pool corruption bug to me.
> The stack trace posted in https://svn.haxx.se/users/archive-2014-05/.shtml
> points towards an endless loop in apr_pool_cleanup_kill().
> The trace ends at APR-util's misc/apr_reslist.c:apr_reslist_cleanup_order_set,
> and of the functions this calls only apr_pool_cleanup_kill() contains loops.
>
> I could not do any further debugging since I only had a production setup
> to look at, which is stable with the workaround 'LDAPSharedCacheSize 0'.
> I also do not have a Windows dev environment and I don't plan on digging
> any further.
>
> Until the real bug gets found and fixed, I would recommend making the
> known workaround the default on Windows. Because the winnt MPM runs a
> single process, there is no benefit to a shared memory cache anyway.
>
> Should I commit this patch?
>
> Index: modules/ldap/util_ldap.c
> ===
> --- modules/ldap/util_ldap.c(revision 1803972)
> +++ modules/ldap/util_ldap.c(working copy)
> @@ -2815,7 +2815,17 @@ static void *util_ldap_create_config(apr_pool_t *p
>  apr_thread_mutex_create(>mutex, APR_THREAD_MUTEX_DEFAULT, st->pool);
>  #endif
>
> +#ifdef WIN32
> +/* XXX The shared memory cache can cause an endless loop on Windows.
> + * See https://svn.haxx.se/users/archive-2014-05/.shtml and
> + * similar reports elsewhere which recommend 'LDAPSharedCacheSize 0'
> + * as a workaround.
> + * Because the winnt MPM uses a single process a shared cache is
> + * not needed anyway so leave it disabled by default. */
> +st->cache_bytes = 0;
> +#else
>  st->cache_bytes = 50;
> +#endif
>  st->search_cache_ttl = 6;
>  st->search_cache_size = 1024;
>  st->compare_cache_ttl = 6;



-- 
Eric Covener
cove...@gmail.com

Re: svn commit: r1824592 - /httpd/httpd/branches/2.4.x/STATUS

[Ruediger Pluem]

On 02/19/2018 07:50 PM, Joe Orton wrote:
> On Sat, Feb 17, 2018 at 02:06:20PM -, minf...@apache.org wrote:
>> Author: minfrin
>> Date: Sat Feb 17 14:06:20 2018
>> New Revision: 1824592
>>
>> URL: http://svn.apache.org/viewvc?rev=1824592=rev
>> Log:
>> Update proposal with fix for rpluem/jorton.
> 
> Extending dav_resource still breaks binary backards compat with all(?) 
> consumers of this API, or am I missing something here?

I don't think so. Modules that just work with pointers to structs created by 
mod_dav shouldn't
have any issues, but see below.

> 
> Look at what mod_dav_svn does with the struct for an example: 
> https://svn.apache.org/viewvc/subversion/trunk/subversion/mod_dav_svn/repos.c?view=markup#l80

I guess this gets us back to this _copy / _sizeof discussion we had in another 
thread recently.
Creating structures / allocating memory for them in modules is IMHO a recipe to 
fail with our current
backporting policy as we allow to extend structures at the end.

Regards

Rüdiger

Re: svn commit: r1824592 - /httpd/httpd/branches/2.4.x/STATUS

[Joe Orton]

On Sat, Feb 17, 2018 at 02:06:20PM -, minf...@apache.org wrote:
> Author: minfrin
> Date: Sat Feb 17 14:06:20 2018
> New Revision: 1824592
> 
> URL: http://svn.apache.org/viewvc?rev=1824592=rev
> Log:
> Update proposal with fix for rpluem/jorton.

Extending dav_resource still breaks binary backards compat with all(?) 
consumers of this API, or am I missing something here?

Look at what mod_dav_svn does with the struct for an example: 
https://svn.apache.org/viewvc/subversion/trunk/subversion/mod_dav_svn/repos.c?view=markup#l80

If breaking backwards compat for module API is fine for 2.4 (and I'm 
still generally on the fence about this), I think it should at least be 
a "hard" break, e.g. by s/dav_register_provider/dav_register_provider2/.  
That way incompatible module combinations break at LoadModule time 
rather than with segfaults.

Regards, Joe

> 
> Modified:
> httpd/httpd/branches/2.4.x/STATUS
> 
> Modified: httpd/httpd/branches/2.4.x/STATUS
> URL: 
> http://svn.apache.org/viewvc/httpd/httpd/branches/2.4.x/STATUS?rev=1824592=1824591=1824592=diff
> ==
> --- httpd/httpd/branches/2.4.x/STATUS (original)
> +++ httpd/httpd/branches/2.4.x/STATUS Sat Feb 17 14:06:20 2018
> @@ -180,6 +180,13 @@ PATCHES PROPOSED TO BACKPORT FROM TRUNK:
>   minfrin: 2.4.x patch not applying for me, trunk patches work
>   icing: fixed the patch to include complete new files instead of svn's 
> internal optimizations
>  
> +  *) mod_dav: Allow other modules to become providers and add ACLs
> + to the DAV response. Requires a release of apr-util v1.6.
> + trunk patch: http://svn.apache.org/r1748322
> +  http://svn.apache.org/r1824590
> + 2.4.x: trunk works modulo CHANGES/MMN
> + +1: minfrin
> +
>  
>  PATCHES/ISSUES THAT ARE BEING WORKED
>[ New entries should be added at the START of the list ]
> @@ -255,23 +262,6 @@ PATCHES/ISSUES THAT ARE BEING WORKED
>   make it nonblocking (by default)?
>  jim: Non-blocking seems the best way to handle...
>  
> -   * mod_dav: Allow other modules to become providers and add ACLs
> - to the DAV response. Requires a release of apr-util v1.6.
> - trunk patch: http://svn.apache.org/r1748322
> - 2.4.x: trunk works modulo CHANGES/MMN
> - +1: minfrin
> - rpluem asks: Will this compile with apr-util < v1.6 and keep
> - mod_dav working (without the new features of the patch of course)?
> - I doubt that we will require apr-util 1.6 for the lifetime of 2.4.x
> - (see the discussion around ap_cstr_casecmp[n] an apr 1.6)
> - minfrin: Yes, as you can see in the patch everything applies only
> - if APR_XML_X2T_PARSED is defined, and the patch was tested with
> - both apr-util v1.6 and apr-util v1.5.
> - -1: jorton, breaks binary backwards compat per dev@ discuss
> - msgid <20160902120654.ga12...@redhat.com>
> - (& also, making the structure change with apr-util version
> - means it breaks binary compat across an apr-util upgrade?)
> -
> * Support PCRE2 (10.x) in place of PCRE (8.x).
>   Submitted by: wrowe, Petr Pisar [ppisar redhat.com]
>   trunk patches:
> 
> 

Re: svn commit: r1824751 [1/2] - in /httpd/httpd/branches/2.4.x: ./ docs/manual/ docs/manual/mod/ include/

[Yann Ylavic]

On Mon, Feb 19, 2018 at 5:27 PM, Yann Ylavic  wrote:
> On Mon, Feb 19, 2018 at 4:47 PM, Jim Jagielski  wrote:
>>
>>
>>> On Feb 19, 2018, at 10:33 AM, Ruediger Pluem  wrote:

>>>
>>> I guess you used your release procedures / scripts for rolling the release. 
>>> So some comments :-)
>>>
>>
>> Where are these? I don't see them in
>>
>> https://svn.apache.org/repos/asf/httpd/site/trunk/tools
>
> This directory is possibly missing a REAME for the exact procedure
> (specific to httpd):
> - make docs
> - pre changes/commits in 2.4.x
> - which script in which order to T
> - post changes/commits in 2.4.x
> ... anything missing ...
>
> Preferably by an experienced RM :)
> That'd certainly help newbies.

Hmm, maybe https://httpd.apache.org/dev/release.html#how-to-do-a-release
is what I was looking for...

Re: svn commit: r1824751 [1/2] - in /httpd/httpd/branches/2.4.x: ./ docs/manual/ docs/manual/mod/ include/

[Yann Ylavic]

On Mon, Feb 19, 2018 at 4:47 PM, Jim Jagielski  wrote:
>
>
>> On Feb 19, 2018, at 10:33 AM, Ruediger Pluem  wrote:
>>>
>>
>> I guess you used your release procedures / scripts for rolling the release. 
>> So some comments :-)
>>
>
> Where are these? I don't see them in
>
> https://svn.apache.org/repos/asf/httpd/site/trunk/tools

This directory is possibly missing a REAME for the exact procedure
(specific to httpd):
- make docs
- pre changes/commits in 2.4.x
- which script in which order to T
- post changes/commits in 2.4.x
... anything missing ...

Preferably by an experienced RM :)
That'd certainly help newbies.

Re: [VOTE] Release httpd-2.4.30

[Jim Jagielski]

> On Feb 19, 2018, at 10:47 AM, Graham Leggett  wrote:
> 
> On 19 Feb 2018, at 5:45 PM, Jim Jagielski  > wrote:
> 
>> Hmmm... I'm not seeing the patch where AP_SERVER_DEVBUILD_BOOLEAN
>> in ap_release.h is set to 0
>> 
>> How does your release process work? What we've always
>> done is make the req changes to the branch and then copy
>> from that branch to the tag. So the tag itself must refer to
>> a specific SVN number on the http-2.4 branch but I'm
>> not seeing where that is done.
> 
> What I've done in the past is follow the commits for previous releases, using 
> them as a template for the steps to take.
> 

+1

Re: [Patch] proof of concept - teaching hooks to yield where sensible

[Jim Jagielski]

Would it be possible to leverage something like libdill (http://libdill.org) 
for that?

> On Feb 19, 2018, at 5:24 AM, Graham Leggett  wrote:
>> 
> 
> This is the problem I want to solve - I want to be able to run multiple 
> connections, and allow them to yield to each other.
> 

Re: svn commit: r1824751 [1/2] - in /httpd/httpd/branches/2.4.x: ./ docs/manual/ docs/manual/mod/ include/

[Jim Jagielski]

> On Feb 19, 2018, at 10:33 AM, Ruediger Pluem  wrote:
>> 
> 
> I guess you used your release procedures / scripts for rolling the release. 
> So some comments :-)
> 

Where are these? I don't see them in

https://svn.apache.org/repos/asf/httpd/site/trunk/tools

Re: [VOTE] Release httpd-2.4.30

[Graham Leggett]

On 19 Feb 2018, at 5:45 PM, Jim Jagielski  wrote:

> Hmmm... I'm not seeing the patch where AP_SERVER_DEVBUILD_BOOLEAN
> in ap_release.h is set to 0
> 
> How does your release process work? What we've always
> done is make the req changes to the branch and then copy
> from that branch to the tag. So the tag itself must refer to
> a specific SVN number on the http-2.4 branch but I'm
> not seeing where that is done.

What I've done in the past is follow the commits for previous releases, using 
them as a template for the steps to take.

Regards,
Graham
—



smime.p7s
Description: S/MIME cryptographic signature

Re: [VOTE] Release httpd-2.4.30

[Jim Jagielski]

Hmmm... I'm not seeing the patch where AP_SERVER_DEVBUILD_BOOLEAN
in ap_release.h is set to 0

How does your release process work? What we've always
done is make the req changes to the branch and then copy
from that branch to the tag. So the tag itself must refer to
a specific SVN number on the http-2.4 branch but I'm
not seeing where that is done.

> On Feb 19, 2018, at 9:54 AM, drugg...@primary.net wrote:
> 
> Hi, all;
>Please find below the proposed release tarball and signatures:
> https://dist.apache.org/repos/dist/dev/httpd/ 
> 
>  
> I would like to call a VOTE over the next few days to release this candidate 
> tarball as 2.4.30:
>  
> [ ] +1: It’s not just good, it’s good enough!
> [ ] +0: Let’s have a talk…
> [ ] -1: There’s trouble in paradise. Here’s what’s wrong.
> -- 
> Daniel Ruggeri

Re: svn commit: r1824751 [1/2] - in /httpd/httpd/branches/2.4.x: ./ docs/manual/ docs/manual/mod/ include/

[Ruediger Pluem]

On 02/19/2018 03:18 PM, drugg...@apache.org wrote:
> Author: druggeri
> Date: Mon Feb 19 14:18:51 2018
> New Revision: 1824751
> 
> URL: http://svn.apache.org/viewvc?rev=1824751=rev
> Log:
> Post 2.4.30 tag updates
> 
> Modified:
> httpd/httpd/branches/2.4.x/CHANGES
> httpd/httpd/branches/2.4.x/STATUS
> httpd/httpd/branches/2.4.x/docs/manual/convenience.map
> httpd/httpd/branches/2.4.x/docs/manual/mod/directives.html.de
> httpd/httpd/branches/2.4.x/docs/manual/mod/directives.html.en
> httpd/httpd/branches/2.4.x/docs/manual/mod/directives.html.es
> httpd/httpd/branches/2.4.x/docs/manual/mod/directives.html.ja.utf8
> httpd/httpd/branches/2.4.x/docs/manual/mod/directives.html.ko.euc-kr
> httpd/httpd/branches/2.4.x/docs/manual/mod/directives.html.tr.utf8
> httpd/httpd/branches/2.4.x/docs/manual/mod/directives.html.zh-cn.utf8
> httpd/httpd/branches/2.4.x/docs/manual/mod/index.html.de
> httpd/httpd/branches/2.4.x/docs/manual/mod/index.html.en
> httpd/httpd/branches/2.4.x/docs/manual/mod/index.html.es
> httpd/httpd/branches/2.4.x/docs/manual/mod/index.html.ja.utf8
> httpd/httpd/branches/2.4.x/docs/manual/mod/index.html.ko.euc-kr
> httpd/httpd/branches/2.4.x/docs/manual/mod/index.html.tr.utf8
> httpd/httpd/branches/2.4.x/docs/manual/mod/index.html.zh-cn.utf8
> httpd/httpd/branches/2.4.x/docs/manual/mod/mod_remoteip.html.en
> httpd/httpd/branches/2.4.x/docs/manual/mod/mod_ssl.html.en
> httpd/httpd/branches/2.4.x/docs/manual/mod/overrides.html.en
> httpd/httpd/branches/2.4.x/docs/manual/mod/quickreference.html.de
> httpd/httpd/branches/2.4.x/docs/manual/mod/quickreference.html.en
> httpd/httpd/branches/2.4.x/docs/manual/mod/quickreference.html.es
> httpd/httpd/branches/2.4.x/docs/manual/mod/quickreference.html.ja.utf8
> httpd/httpd/branches/2.4.x/docs/manual/mod/quickreference.html.ko.euc-kr
> httpd/httpd/branches/2.4.x/docs/manual/mod/quickreference.html.tr.utf8
> httpd/httpd/branches/2.4.x/docs/manual/mod/quickreference.html.zh-cn.utf8
> httpd/httpd/branches/2.4.x/docs/manual/sections.html.tr.utf8
> httpd/httpd/branches/2.4.x/docs/manual/sitemap.html.de
> httpd/httpd/branches/2.4.x/docs/manual/sitemap.html.en
> httpd/httpd/branches/2.4.x/docs/manual/sitemap.html.es
> httpd/httpd/branches/2.4.x/docs/manual/sitemap.html.ja.utf8
> httpd/httpd/branches/2.4.x/docs/manual/sitemap.html.tr.utf8
> httpd/httpd/branches/2.4.x/docs/manual/sitemap.html.zh-cn.utf8
> httpd/httpd/branches/2.4.x/include/ap_release.h

I guess you used your release procedures / scripts for rolling the release. So 
some comments :-)

I miss the increase of the httpd.patch entity in  docs/manual/style/version.ent.

> 
> Modified: httpd/httpd/branches/2.4.x/CHANGES
> URL: 
> http://svn.apache.org/viewvc/httpd/httpd/branches/2.4.x/CHANGES?rev=1824751=1824750=1824751=diff
> ==
> --- httpd/httpd/branches/2.4.x/CHANGES [utf-8] (original)
> +++ httpd/httpd/branches/2.4.x/CHANGES [utf-8] Mon Feb 19 14:18:51 2018
> @@ -1,6 +1,8 @@
>   -*- coding: utf-8 
> -*-
>  Changes with Apache 2.4.30
>  
> +Changes with Apache 2.4.30
> +

I guess you want to add a 'Changes with Apache 2.4.31'


>*) mod_proxy: Provide an RFC1035 compliant version of the hostname in the
>   proxy_worker_shared structure. PR62085 [Graham Leggett]
>  
> 
> Modified: httpd/httpd/branches/2.4.x/STATUS
> URL: 
> http://svn.apache.org/viewvc/httpd/httpd/branches/2.4.x/STATUS?rev=1824751=1824750=1824751=diff
> ==
> --- httpd/httpd/branches/2.4.x/STATUS (original)
> +++ httpd/httpd/branches/2.4.x/STATUS Mon Feb 19 14:18:51 2018
> @@ -29,7 +29,7 @@ Release history:
>  [NOTE that x.{odd}.z versions are strictly Alpha/Beta releases,
>while x.{even}.z versions are Stable/GA releases.]
>  
> -2.4.30  : In development. Jim proposes a T Feb. 19, 2018.
> +-8s2.4.30  : In development. Jim proposes a T Feb. 19, 2018.

I guess you want to mark 2.4.30 as tagged.

>  2.4.29  : Tagged on October 17, 2017. Released on October 23, 2017.
>  2.4.28  : Tagged on September 25, 2017. Released on October 5, 2017.
>  2.4.27  : Tagged on July 6, 2017. Released on July 11, 2017.

Regards

Rüdiger

Re: [VOTE] Release httpd-2.4.30

[Stefan Eissing]

> Am 19.02.2018 um 15:54 schrieb  :
> 
> Hi, all;
>Please find below the proposed release tarball and signatures:
> https://dist.apache.org/repos/dist/dev/httpd/
>  
> I would like to call a VOTE over the next few days to release this candidate 
> tarball as 2.4.30:
>  
> [ ] +1: It’s not just good, it’s good enough!
> [ ] +0: Let’s have a talk…
> [ ] -1: There’s trouble in paradise. Here’s what’s wrong.
> -- 
> Daniel Ruggeri

+1 

Tested: 17.4.0 Darwin (High Sierra 10.13.3), event+worker, http2 und md

Thanks for RMing, Daniel!

[VOTE] Release httpd-2.4.30

[DRuggeri]

Hi, all;

   Please find below the proposed release tarball and signatures:

https://dist.apache.org/repos/dist/dev/httpd/

 

I would like to call a VOTE over the next few days to release this candidate
tarball as 2.4.30:

 

[ ] +1: It's not just good, it's good enough!

[ ] +0: Let's have a talk.

[ ] -1: There's trouble in paradise. Here's what's wrong.

-- 

Daniel Ruggeri

 

Apache httpd 2.4.30-dev available for testing

[Steffen]

The Windows lovers can test the soon coming httpd 2.4.30.

See www.apachelounge.com/viewtopic.php?p=36509

Please let me us know how it goes.

Issues you can report here or at Apache Lounge.

Re: Current branche 2.4.30-dev issues

[Steffen]

Added  mod_proxy_uwsgi to installwinconf.awk and BasAddr.ref in trunk 
and branches.



On 18-2-2018 14:44, Steffen wrote:


The bad one:

-- mod_remoteip.obj : error LNK2019: unresolved external symbol 
__imp__ntohs@4 referenced in function _remoteip_input_filter


Can overcome :

--  missing in win build files, only we have mod_proxy_uwsgi.dsp

-- not sure, mod_md; should curl and jansson be added to 
notice/license files ?


Not using here :

-- mod_md missing in cmake

Re: [Patch] proof of concept - teaching hooks to yield where sensible

[Graham Leggett]

On 19 Feb 2018, at 11:45 AM, Yann Ylavic  wrote:

> We already have some event's mechanism(s) be async at the handler
> layer (write completion, callbacks in trunk).
> The "common connection state/milestone" proposal looks interesting for
> compatibility (maybe add new CONN_STATEs instead of the similar
> milestone type?).

The milestones are completely independent of the connection state, they simply 
mean “in this module, in this function, we have successfully got this far, if a 
hook returned AGAIN, continue from here and do not repeat hooks that have 
completed successfully”.

> However the most challenging part seems to be in/output filters part,
> they currently handle EAGAIN like a real error (logging, brigade
> cleanup, ...) and usually don't expect to be called again.

This is a solved problem in trunk.

It turned out the network filter was async already, and had the ability to be 
called again. In the async filter patches I abstracted this behaviour out and 
taught other filters how to do this - most notably mod_ssl.

This problem doesn’t even need to be solved for this to work - if a module 
chooses not to return AGAIN, that’s completely fine.

> We can surely address all the cases upstream, what about third parties?

Third party modules just work like they have before, and the behaviour is the 
same as before - a slot is consumed until done.

Regards,
Graham
—



smime.p7s
Description: S/MIME cryptographic signature

Re: [Patch] proof of concept - teaching hooks to yield where sensible

[Stefan Eissing]

> Am 19.02.2018 um 11:24 schrieb Graham Leggett :
> 
> On 19 Feb 2018, at 11:14 AM, Stefan Eissing  
> wrote:
> 
>> If I understand your gist correctly, this would allow HTTP/2 processing to 
>> return to the main (async) event loop more often. Which would be great.
>> 
>> In the case of HTTP/2, it would be even more cool, to trigger the 
>> (re-)processing of an AGAIN connection from another thread. The use
>> case is: H2 main look started request, awaits response HEADERS/DATA *or* 
>> incoming data from client.
>> 
>> now: timed wait on a condition with read checks on the main connection at 
>> dynamic intervals
>> then: return AGAIN (READ mode) to event look, new HEADERS/DATA from request 
>> triggers re-process_connection.
> 
> This is the problem I want to solve - I want to be able to run multiple 
> connections, and allow them to yield to each other.
> 
> I want to give our hooks the option to bite off and process data in chunks 
> they’re in control of. Right now, you call the handler hook and it’s a one 
> shot deal - the handler must finish whatever it wants to do, and only return 
> once done.
> 
> Yes, in most cases our handlers generate data and pass them to the filter 
> chain, which then handles the async data flow, but it would be nice if they 
> weren’t forced to do that.

+1

Re: [Patch] proof of concept - teaching hooks to yield where sensible

[Graham Leggett]

On 19 Feb 2018, at 11:14 AM, Stefan Eissing  
wrote:

> If I understand your gist correctly, this would allow HTTP/2 processing to 
> return to the main (async) event loop more often. Which would be great.
> 
> In the case of HTTP/2, it would be even more cool, to trigger the 
> (re-)processing of an AGAIN connection from another thread. The use
> case is: H2 main look started request, awaits response HEADERS/DATA *or* 
> incoming data from client.
> 
> now: timed wait on a condition with read checks on the main connection at 
> dynamic intervals
> then: return AGAIN (READ mode) to event look, new HEADERS/DATA from request 
> triggers re-process_connection.

This is the problem I want to solve - I want to be able to run multiple 
connections, and allow them to yield to each other.

I want to give our hooks the option to bite off and process data in chunks 
they’re in control of. Right now, you call the handler hook and it’s a one shot 
deal - the handler must finish whatever it wants to do, and only return once 
done.

Yes, in most cases our handlers generate data and pass them to the filter 
chain, which then handles the async data flow, but it would be nice if they 
weren’t forced to do that.

Regards,
Graham
—



smime.p7s
Description: S/MIME cryptographic signature

Re: svn commit: r1824465 - /httpd/httpd/patches/2.4.x/mod_ssl_policy.diff

[Stefan Eissing]

> Am 19.02.2018 um 10:54 schrieb Yann Ylavic :
> 
> On Mon, Feb 19, 2018 at 10:42 AM, Stefan Eissing
>  wrote:
>> After pondering your comments and questions a bit over the weekend, I 
>> decided to
>> withdraw the backport proposal for 2.4.x. Instead, I will simplify SSLPolicy 
>> in
>> trunk and propose a backport for the next release.
>> 
>> My current thinking is to get rid of "" and just introduce
>> a fixed "SSLPolicy modern|intermediate|old" which is updated from the Mozilla
>> definitions of these terms (a script for that is already in modules/ssl). 
>> This
>> will only apply to the client facing SSL properties.
>> 
>> "SSLPolicy" will then just act as a normal SSL configuration directive, that
>> sets a defined number of parameters. Those parameters will get updated in our
>> releases (and by distros if they want to update a LTS version with a more 
>> secure
>> setting).
>> 
>> If can be overridden by site admins, just like any other directive. The 
>> configuration
>> 
>>   SSLProtocol all
>>   SSLPolicy modern
>> 
>> would just enable TLSv1.2 (and newer), while
>> 
>>   SSLPolicy modern
>>   SSLProtocol +TLSv1.3
>> 
>> would override it.
> 
> Looks good to me.
> 
> The all in one defined policy is interesting still, let's take the
> time to think more about it.
> It could possibly be implemented as macros too, no?

*Anything* can be implemented as a macro... ;-)

SSLPolicy does *not* mean: "compression off, protocols x y z, ciphers 
blabla-256"
SSLPolicy means: "what your Apache provider considers best SSL settings for 
modern browsers"

If you build from source from httpd.apache.org, it is what Mozilla considers 
modern
at the time of our release. If you get it from a distro, it means what they 
think relevant.
They could update it in years to come for an Apache in their LTS line.

By doing it as a new directive, we enable good advice:

"If you have an Apache > 2.4.30, use the following to configure your https: 
site."

Instead of:

"If you have an Apache > 2.4.30 with the standard macro definitions included 
and mod_macro
enabled, use the following... Btw. do not edit the macro definitions or your 
settings will
no longer auto-update. But if you do, check after every update also, if the 
macros are still
as you want them, because your changes may also have been overwritten."

The macro approach will not work for an internet that will be 99% https: very, 
very soon, IMO.

Cheers, Stefan

Re: svn commit: r1824465 - /httpd/httpd/patches/2.4.x/mod_ssl_policy.diff

[Yann Ylavic]

On Mon, Feb 19, 2018 at 10:42 AM, Stefan Eissing
 wrote:
> After pondering your comments and questions a bit over the weekend, I decided 
> to
> withdraw the backport proposal for 2.4.x. Instead, I will simplify SSLPolicy 
> in
> trunk and propose a backport for the next release.
>
> My current thinking is to get rid of "" and just introduce
> a fixed "SSLPolicy modern|intermediate|old" which is updated from the Mozilla
> definitions of these terms (a script for that is already in modules/ssl). This
> will only apply to the client facing SSL properties.
>
> "SSLPolicy" will then just act as a normal SSL configuration directive, that
> sets a defined number of parameters. Those parameters will get updated in our
> releases (and by distros if they want to update a LTS version with a more 
> secure
> setting).
>
> If can be overridden by site admins, just like any other directive. The 
> configuration
>
>SSLProtocol all
>SSLPolicy modern
>
> would just enable TLSv1.2 (and newer), while
>
>SSLPolicy modern
>SSLProtocol +TLSv1.3
>
> would override it.

Looks good to me.

The all in one defined policy is interesting still, let's take the
time to think more about it.
It could possibly be implemented as macros too, no?

Thanks,
Yann.

Re: [Patch] proof of concept - teaching hooks to yield where sensible

[Yann Ylavic]

On Sun, Feb 18, 2018 at 12:11 AM, Graham Leggett  wrote:>
> As an extension to the idea of filters being async and being able to
> yield and break up their work into small chunks, I am keen to extend
> that idea to selected hooks.
>
> The patch below is a proof of concept, showing what it might look
> like if the pre_connection and process_connection hooks were able to
> return the code AGAIN. This means “please call me again”.
>
> In implementing this, we would start at the outermost hook, and then
> work inwards. We would also need to change these void functions to
> return ints, so that AGAIN could bubble upwards. Eventually the
> handler hook will be able to return AGAIN.
>
> Any module that doesn’t yield just behaves as modules do now.

We already have some event's mechanism(s) be async at the handler
layer (write completion, callbacks in trunk).
The "common connection state/milestone" proposal looks interesting for
compatibility (maybe add new CONN_STATEs instead of the similar
milestone type?).
However the most challenging part seems to be in/output filters part,
they currently handle EAGAIN like a real error (logging, brigade
cleanup, ...) and usually don't expect to be called again.

We can surely address all the cases upstream, what about third parties?


Regards,
Yann.

Re: svn commit: r1824465 - /httpd/httpd/patches/2.4.x/mod_ssl_policy.diff

[Stefan Eissing]

After pondering your comments and questions a bit over the weekend, I decided to
withdraw the backport proposal for 2.4.x. Instead, I will simplify SSLPolicy in
trunk and propose a backport for the next release.

My current thinking is to get rid of "" and just introduce
a fixed "SSLPolicy modern|intermediate|old" which is updated from the Mozilla
definitions of these terms (a script for that is already in modules/ssl). This 
will only apply to the client facing SSL properties.

"SSLPolicy" will then just act as a normal SSL configuration directive, that
sets a defined number of parameters. Those parameters will get updated in our
releases (and by distros if they want to update a LTS version with a more secure
setting). 

If can be overridden by site admins, just like any other directive. The 
configuration

   SSLProtocol all
   SSLPolicy modern

would just enable TLSv1.2 (and newer), while

   SSLPolicy modern
   SSLProtocol +TLSv1.3

would override it.

Cheers,

Stefan


> Am 17.02.2018 um 18:20 schrieb Stefan Eissing :
> 
>> 
>> Am 16.02.2018 um 17:34 schrieb Yann Ylavic :
>> 
>> Hi Stefan,
>> 
>> On Fri, Feb 16, 2018 at 2:05 PM,   wrote:
>>> Author: icing
>>> Date: Fri Feb 16 13:05:27 2018
>>> New Revision: 1824465
>>> 
>>> URL: http://svn.apache.org/viewvc?rev=1824465=rev
>>> Log:
>>> sslpolicy patch for 2.4.x
>> 
>> Quick incomplete/review, two question...
> 
> Thanks for reviewing.
> 
>> There seems to be predefined/moz policies with "SSLProxyVerify
>> require" enforced, how can it work w/o a CA? That impose also setting
>> SSLProxyCACertificate* for httpd to start/work, right?
> 
> If it only works with an explicit CA file, then it should not be
> on. I think I was hoping that openssl tied into some CA vault on
> its own and a CA file was only necessary for non-public certs.
> 
>>  is global only, but since it's not a real
>> directory/section itself anything can nest inside (including another
>> )?
> 
> Any non-SSL directive inside would not be part of it. That could
> lead to confusion, since other directives will not see the difference
> and will not complain. I am not sure if introducing new section types,
> which adds complexity in the server is worth it. We could add a section
> to the documentation that states this limitation more clearly.
> 
> Also, nesting  should be prevented, as it will
> not give the results that one would expect (I have not tested that,
> but I think it would behave as if the two were side by side).
> 
> Cheers,
> 
> Stefan

Re: [Patch] proof of concept - teaching hooks to yield where sensible

[Stefan Eissing]

If I understand your gist correctly, this would allow HTTP/2 processing to 
return to the main (async) event loop more often. Which would be great.

In the case of HTTP/2, it would be even more cool, to trigger the 
(re-)processing of an AGAIN connection from another thread. The use
case is: H2 main look started request, awaits response HEADERS/DATA *or* 
incoming data from client.

 now: timed wait on a condition with read checks on the main connection at 
dynamic intervals
then: return AGAIN (READ mode) to event look, new HEADERS/DATA from request 
triggers re-process_connection.

-Stefan



> Am 18.02.2018 um 00:11 schrieb Graham Leggett :
> 
> Hi all,
> 
> As an extension to the idea of filters being async and being able to yield 
> and break up their work into small chunks, I am keen to extend that idea to 
> selected hooks.
> 
> The patch below is a proof of concept, showing what it might look like if the 
> pre_connection and process_connection hooks were able to return the code 
> AGAIN. This means “please call me again”.
> 
> In implementing this, we would start at the outermost hook, and then work 
> inwards. We would also need to change these void functions to return ints, so 
> that AGAIN could bubble upwards. Eventually the handler hook will be able to 
> return AGAIN.
> 
> Any module that doesn’t yield just behaves as modules do now.
> 
> Does this make sense?
> 
> Regards,
> Graham
> --
> 
> Index: include/http_connection.h
> ===
> --- include/http_connection.h (revision 1824594)
> +++ include/http_connection.h (working copy)
> @@ -105,11 +105,14 @@
>  * This hook gives protocol modules an opportunity to set everything up
>  * before calling the protocol handler.  All pre-connection hooks are
>  * run until one returns something other than ok or decline
> + *
> + * The pre connection hook may trigger a break in processing and ask to
> + * be called again by returning AGAIN.
>  * @param c The connection on which the request has been received.
>  * @param csd The mechanism on which this connection is to be read.
>  *Most times this will be a socket, but it is up to the module
>  *that accepts the request to determine the exact type.
> - * @return OK or DECLINED
> + * @return OK, DECLINED or AGAIN
>  */
> AP_DECLARE_HOOK(int,pre_connection,(conn_rec *c, void *csd))
> 
> @@ -118,8 +121,11 @@
>  * established, the protocol module must read and serve the request.  This
>  * function does that for each protocol module.  The first protocol module
>  * to handle the request is the last module run.
> + *
> + * Protocol modules may trigger a break in processing and ask to be called
> + * again by returning AGAIN.
>  * @param c The connection on which the request has been received.
> - * @return OK or DECLINED
> + * @return OK, DECLINED or AGAIN
>  */
> AP_DECLARE_HOOK(int,process_connection,(conn_rec *c))
> 
> Index: include/httpd.h
> ===
> --- include/httpd.h   (revision 1824594)
> +++ include/httpd.h   (working copy)
> @@ -460,6 +460,8 @@
>  */
> #define SUSPENDED -3 /**< Module will handle the remainder of the request.
>   * The core will never invoke the request again, */
> +#define AGAIN -4/**< Module wants to be called again.
> +  */
> 
> /** Returned by the bottom-most filter if no data was written.
>  *  @see ap_pass_brigade(). */
> Index: server/connection.c
> ===
> --- server/connection.c   (revision 1824594)
> +++ server/connection.c   (working copy)
> @@ -29,6 +29,7 @@
> #include "scoreboard.h"
> #include "http_log.h"
> #include "util_filter.h"
> +#include "core.h"
> 
> APR_HOOK_STRUCT(
> APR_HOOK_LINK(create_connection)
> @@ -207,15 +208,38 @@
> 
> AP_CORE_DECLARE(void) ap_process_connection(conn_rec *c, void *csd)
> {
> +conn_config_t *conn_config = ap_get_core_module_config(c->conn_config);
> int rc;
> +
> +switch (AP_CORE_DEFAULT(conn_config, milestone, CONN_MILESTONE_START)) {
> +case CONN_MILESTONE_START:
> +
> ap_update_vhost_given_ip(c);
> 
> +case CONN_MILESTONE_PRE_CONNECTION:
> +conn_config->milestone = CONN_MILESTONE_START;
> rc = ap_run_pre_connection(c, csd);
> +if (rc == AGAIN) {
> +conn_config->milestone = CONN_MILESTONE_PRE_CONNECTION;
> +return;
> +}
> +
> if (rc != OK && rc != DONE) {
> c->aborted = 1;
> }
> 
> if (!c->aborted) {
> -ap_run_process_connection(c);
> +
> +case CONN_MILESTONE_PROCESS_CONNECTION:
> +conn_config->milestone = CONN_MILESTONE_START;
> +rc = ap_run_process_connection(c);
> +if (rc == AGAIN) {
> +conn_config->milestone = CONN_MILESTONE_PROCESS_CONNECTION;
> +