Re: svn commit: r1893977 - in /httpd/httpd/branches/2.4.x: ./ CHANGES include/ap_mmn.h include/httpd.h server/gen_test_char.c server/request.c server/util.c

2021-10-07 Thread Rainer Jung

Am 07.10.2021 um 14:27 schrieb yla...@apache.org:

Modified: httpd/httpd/branches/2.4.x/include/ap_mmn.h
URL: 
http://svn.apache.org/viewvc/httpd/httpd/branches/2.4.x/include/ap_mmn.h?rev=1893977=1893976=1893977=diff
==
--- httpd/httpd/branches/2.4.x/include/ap_mmn.h (original)
+++ httpd/httpd/branches/2.4.x/include/ap_mmn.h Thu Oct  7 12:27:43 2021
@@ -579,6 +579,9 @@
   *   ap_proxy_define_worker_ex() to mod_proxy.h
   * 20120211.116 (2.4.49-dev) add conn_rec->outgoing and ap_ssl_bind_outgoing()
   * 20120211.117 (2.4.50-dev) Add ap_pre_connection
+ * 20210926.1 (2.5.1-dev)  Add ap_unescape_url_ex() and deprecate
+ * AP_NORMALIZE_DROP_PARAMETERS
+ *
   */
  
  #define MODULE_MAGIC_COOKIE 0x41503234UL /* "AP24" */


Doesn't this need (a cosmetic) adjustment for 2.4.x?

Plus: if a minor bump is needed, this commit contains only a comment change.

Thanks for your intensive work!

Rainer


Re: [VOTE] Release httpd-2.4.50-rc1 as httpd-2.4.50

2021-10-02 Thread Rainer Jung
Thanks for testing Dennis. We need to get this release out quick due to 
regressions, so it wasn't the right moment to apply the OpenSSL patch.


I'm confident, that Joe's OpenSSL 3.0.0 patch will be included in the 
next regular 2.4 release.


Best regards,

Rainer

Am 02.10.2021 um 13:01 schrieb Dennis Clarke:

On 10/1/21 10:40, ste...@eissing.org wrote:

Hi, all;
Please find below the proposed release tarball and signatures:
https://dist.apache.org/repos/dist/dev/httpd/

I would like to call a VOTE over the next few days to release
this candidate tarball httpd-2.4.50-rc1 as 2.4.50:
[ ] +1: It's not just good, it's good enough!
[ ] +0: Let's have a talk.
[ ] -1: There's trouble in paradise. Here's what's wrong.

The computed digests of the tarball up for vote are:
sha1: afac1bf6aaa84ea2878c56ed56a49f5efdd7ff73 *httpd-2.4.50-rc1.tar.gz
sha256: feb87f9cc60e02782d795d30cd60a36e918c82fe9a2e7363b0ae28a78be9613a 
*httpd-2.4.50-rc1.tar.gz
sha512: 
3b5e06d8428f14bae8173dbb27921496831c7c14c31850d2df0c0d6f69e931bbf0e4e71c6a250b4f5c0d646d1b7da813bbe61711f8a79ab5f80d39dfd176f57c
 *httpd-2.4.50-rc1.tar.gz



+1: It's not just good, it's good with OpenSSL 1.1.1

On ye SPARC64 Solaris 10 server here it seems to work just fine with
OpenSSL 1.1.1l and of course it does not work ( yet ) with OpenSSL 3.0.0
and we already know that :

beta # /opt/bw/bin/apachectl start
httpd: Syntax error on line 99 of /opt/bw/etc/apache/httpd/httpd.conf:
Cannot load modules/mod_ssl.so into server: ld.so.1: httpd: fatal:
relocation error: file /opt/bw/modules/mod_ssl.so: symbol ERR_GET_FUNC:
referenced symbol not found
beta #

I don't know if trunk is functional and I have not really jumped on
trunk in a few weeks and I'll give that a whirl but autoconf/autoreconf
autotools madness was giving me hardship. At least last time I looked
which was 1893292 about two weeks ago.

Anyways 2.4.50 seems to work just fine with OpenSSL 1.1.1l for me.


Re: sending announcement mail

2021-09-18 Thread Rainer Jung

Am 16.09.2021 um 13:59 schrieb ste...@eissing.org:

Am 16.09.2021 um 13:57 schrieb ste...@eissing.org:

Am 16.09.2021 um 13:50 schrieb ste...@eissing.org:

Am 16.09.2021 um 13:46 schrieb Daniel Gruno :

it's on announce@httpd.a.o already. I modded it through.

You need to send it to annou...@apache.org as well.


I got the mail from annou...@apache.org, but not from annou...@httpd.apache.org

Alternate realities?


Hmm, maybe my MacOS mail client confuses me...


I give up. Could you mail 
 to the 
list where it missing?


I think there is still no announcement on annou...@apache.org. At least 
I don't see it here


https://mail-archives.apache.org/mod_mbox/www-announce/202109.mbox/browser

and I think I also didn't get it. For the previous 2.4.48 there was one 
from Christophe on June 1st:


https://mail-archives.apache.org/mod_mbox/www-announce/202106.mbox/browser

Thanks and regards,

Rainer


Re: trunk/rc usable with OpenSSL 3.0.0 ?

2021-09-13 Thread Rainer Jung

Hi Dennis,

Am 13.09.2021 um 11:05 schrieb Dennis Clarke:

On 9/13/21 04:22, Joe Orton wrote:

On Mon, Sep 13, 2021 at 01:23:37AM -0400, Dennis Clarke wrote:


ALL :


I may receive no reply to this but in general I have been able to build
Apache httpd from any release tarball as well as from trunk. When httpd
needed to get TLS 1.3 working it was a slam dunk to get that working and
it did. However now we have OpenSSL 3.0.0 and it seems that neither the
latest RC works nor does trunk.

So then ... how to proceed ?


What fails with trunk?

It's expected that httpd 2.4 doesn't support 3.0 yet, hopefully we can
get this in for a future release but OpenSSL 3.0 has been a moving
target until just six days ago.

Regards, Joe



Why "expected" that httpd 2.4 doesn't support 3.0 ?


"expected" in the sense that the httpd project developers know about 
this. So "we" expect it.



While I realize that 3.0.0 is very shiney new and still has a green glow
to is we also know that the beta program has been in place for months
and the release candidates go back a year.


We did successfully test 3.0.0 alpha and beta in combination with the 
previous 2.4 releases. See for instance my release vote mails then.


3.0.0 use in combination with httpd 2.4.x did only break recently, due 
to changes in 3.0.0 that were not part of earlier alpha and beta 
releases. That's why we only recently got aware of needed mod_ssl 
changes to again make it work with 3.0.0. As mentioned by others the 
2.4.49 release is important for other reasons and we do not want to 
break it due to last minute mod_ssl changes, which would only be useful 
for a minority of users. Most would not yet go with OpenSSL 3.0.0.


Joe (Orton) has provided a pull request for 2.4.x based on httpd trunk 
to again support OpenSSL 3.0.0 and that's why he is interested in your 
observed httpd trunk failures with 3.0.0.



You have me at a loss.


Hopefully our situation is now understandable again?


That Apache httpd, the biggest web server on planet Earth ( let me check
mars ) has never looked at OpenSSL 3.0.0 as an event in the mail? It has
been shipped. Delivered. Done. It works. What are you saying?


We - for instance me - look at it since quite some time. The breaks were 
introduced recently in OpenSSL land. That's why we need a few weeks to 
react.


Thanks for caring about httpd in Solaris land!

Regards,

Rainer


APR 1.7.1 release?

2021-08-31 Thread Rainer Jung

Hi there,

any chance we find an RM for a APR 1.7.1 release? At least there was the 
fix for CVE-2021-35940 and CHANGES contains 15 more items (many of them 
platform specific or build improvements). Last release 1.7.0 was in 
April 2019.


For APR-util I don't know the current state and release needs for the 
1.6.x and 1.7.x branches. Last 1.6.x release was in October 2017, 1.7.x 
has never been released. CHANGES for 1.6.x only contains one 
apr_dbm_gdbm fix plus a minor libtool use improvement.


Apache httpd is planing to start a release cycle soon and it would be 
nice to have a clean APR 1.7.1 and maybe APR-util also.


Thanks and regards,

Rainer


Re: Late(r) stop of children processes on restart

2021-08-25 Thread Rainer Jung

Thanks for the headroom explanation Yann, good reading!

Rainer

Am 25.08.2021 um 13:23 schrieb Yann Ylavic:

On Tue, Jun 29, 2021 at 3:00 PM Rainer Jung  wrote:


Am 29.06.2021 um 14:31 schrieb Stefan Eissing:

Can comment really on the diff, but totally agree on the goal to minimize the 
unresponsive time and make graceful less disruptive.

So +1 for that.


+1 on the intention as well.


Checked in trunk (r1892587 + r1892595).



Not sure, whether that means people would need more headroom in the
scoreboard (which would probably warrant a sentence in CHANGES or docs
about that) or whether it just means the duration during which that
headroom is used changes (which I wouldn't care about).


The restart delay between stop and start is now minimal (no reload in
between), but the headroom needed does not change AIUI.
We still have the situation where connections (worker threads) are
active for both the new and old generations of children processes, and
its duration depends mainly on the actual lifetime of the connections.
So the current tunings still hold I think.

What changes now is that for both graceful and ungraceful restarts the
main process fully consumes one CPU (to reload) while children are
actively running (the old generation keeps accepting/processing
connections during reload), whereas before the children were tearing
down thus easing the CPUs (but filling the sockets backlogs,
potentially until exhaustion..).
So there might be a greater load spike (overall) than before on reload.

A note on the headroom while at it:
mpm_event is possibly less consumer of children (hence scoreboard
slots) on restart, because when a child is dying it stops (and thus
doesn't account for) the worker threads above the remaining number of
connections, which will accurately create children of the new
generation to scale. mpm_worker never stops threads (this improvement
never made it there AFAICT), thus by accounting for inactive threads
as active it will finally create more children of the new generation
as connections arrive (eventually reaching the limits earlier, or
blocking/waiting for worker threads in the new generation of children
overflowed by incoming connections which the main process thinks are
evenly distributed across all the children, including old
generation's).
I don't know how hard/worthy it is to align mpm_worker with mpm_event
on this, just a note..


Cheers;
Yann.


Re: Late(r) stop of children processes on restart

2021-06-29 Thread Rainer Jung

Am 29.06.2021 um 14:31 schrieb Stefan Eissing:

Can comment really on the diff, but totally agree on the goal to minimize the 
unresponsive time and make graceful less disruptive.

So +1 for that.


+1 on the intention as well.

Not sure, whether that means people would need more headroom in the 
scoreboard (which would probably warrant a sentence in CHANGES or docs 
about that) or whether it just means the duration during which that 
headroom is used changes (which I wouldn't care about).


Thanks and regards,

Rainer


Am 28.06.2021 um 16:25 schrieb Yann Ylavic :

When the MPM event/worker is restarting, it first signals the
children's processes to stop (via POD), then reload the configuration,
and finally start the new generation.

This may be problematic when the reload takes some time to complete
because incoming connections are no longer processed.
A module at day $job is loading quite some regexes and JSON schemas
for each vhost, and I have seen restarts take tens of seconds to
complete with a large number of vhosts. I suppose this can happen with
many RewriteRules too.

How about we wait for the reload to complete before stopping the old
generation, like in the attached patch (MPM event only for now,
changes in worker would be quite similar)?

This is achieved by creating the PODs and listeners buckets from a
generation pool (gen_pool), with a different lifetime than pconf.
gen_pool survives restarts and is created/cleared after the old
generation is stopped, entirely in the run_mpm hook, so the stop and
PODs and buckets handling is moved there (most changes are cut/paste).

WDYT?

Regards;
Yann.



Question about APR trunk and httpd ldap modules

2021-05-27 Thread Rainer Jung

Hi there,

is my understanding correct, that even httpd trunk (and then also 2.4.x) 
needs LDAP support in APR/APU to build mod_ldap and mod_authnz_ldap?


So since we removed LDAP support from APR trunk, that means those 
modules currently can not be build using APR trunk, neither in httpd 
trunk nor in httpd 2.4.x. Correct?


Just trying to understand the current situation about APR trunk and its 
implications.


Thanks and regards,

Rainer


Test suite using OpenSSL 0.9.8 client against OpenSSL 3.0.0 server

2021-05-24 Thread Rainer Jung
FYI: the problems I observed when running the httpd test suite using an 
OpenSSL 0.9.8zh based client against a server build using OpenSSL 3.0.0 
originated in the fact, that OpenSSL 3.0.0 by default no longer allows 
RSA SHA1 and DSA SHA1 as signature algorithms. But 0.9.8 only support 
TLS 1.0 which in turn only supports these signature algorithms. So 
although 3.0.0 still supports TLS 1.0, with a default configuration such 
handshakes no longer work (which is a good thing, except for this testing).


The fix to make the test suite pass was to add "@SECLEVEL=0" to the end 
of the CipherSuite lines in t/conf/ssl/ssl.conf and t/conf/http2.conf 
(directly attached to the end of the existing config values).


Then the handshakes complete and the tests behave as expected, e.g. the 
http2 test no longer complains about the wrong number of tests and skips 
those needing SNI and ALPN etc.


Of course this addition doesn't make sense as a general change to the 
test suite.


One remaining minor nit: 3.0.0 has narrowed down the allowed DH 
parameters. I think that's the reason I sporadically get not strictly 
reproducible faults logged in the server as:


[Mon May 24 11:15:23.944579 2021] [ssl:trace3] [pid 3787] 
ssl_engine_kernel.c(2223): [client 127.0.0.1:36883] OpenSSL: Write: error
[Mon May 24 11:15:23.944586 2021] [ssl:trace3] [pid 3787] 
ssl_engine_kernel.c(2242): [client 127.0.0.1:36883] OpenSSL: Exit: error 
in error
[Mon May 24 11:15:23.944611 2021] [ssl:info] [pid 3787] [client 
127.0.0.1:36883] AH02008: SSL library error 1 in handshake (server 
localhost:8564)
[Mon May 24 11:15:23.944635 2021] [ssl:info] [pid 3787] SSL Library 
Error: error:02800066:Diffie-Hellman routines::invalid public key ()
[Mon May 24 11:15:23.944647 2021] [ssl:info] [pid 3787] SSL Library 
Error: error:0A0C0103:SSL routines::internal error ()
[Mon May 24 11:15:23.944655 2021] [ssl:info] [pid 3787] [client 
127.0.0.1:36883] AH01998: Connection closed to child 7 with abortive 
shutdown (server localhost:8564)


Especially tests in t/ssl/proxy.t fail every now and then, because that 
test file makes a lot of connections. Very rarely also t/ssl/fakeauth.t.


These sporadic failures seem not to occur with OpenSSL up to 1.1.1k-1 in 
the server and also not with OpenSSL 1.0.1 and later in the client.


Regards,

Rainer


Re: [VOTE] Release httpd-2.4.48

2021-05-23 Thread Rainer Jung

Am 17.05.2021 um 23:36 schrieb Christophe JAILLET:

Hi, all;
    Please find below the proposed release tarball and signatures:
https://dist.apache.org/repos/dist/dev/httpd/

I would like to call a VOTE over the next few days to release this 
candidate tarball as 2.4.48:

[X] +1: It's not just good, it's good enough!
[ ] +0: Let's have a talk.
[ ] -1: There's trouble in paradise. Here's what's wrong.

The computed digests of the tarball up for vote are:
sha1: b581bcfdd939fe77c3821f7ad3863c7307374919 *httpd-2.4.48.tar.gz
sha256: 315c0bc50206b866fb17c2cdc28c1973765a8d59ca168b80286e8cb077d0510e 
*httpd-2.4.48.tar.gz
sha512: 
91980f757fc0dede8c6cbf54ed973f82a63098aa50d0fce15fe3537687b4ffbb48ed50cdb4ae14eb4a8703450f032daf73f4f3d5e2dd0f75721948e12a9c6dfb 
*httpd-2.4.48.tar.gz


The SVN tag is '2.4.48' at r1889975.


+1 to release and thanks a bunch for RM!

Summary: all OK except for

- one single crash on SLES 11 statically linked during SSL handshake.

- two crashes on Solaris with prefork MPM during shutdown. Only with 
released APR/APU not with svn heads. Tests ongoing. I think I have seen 
such before, so not a regression.


- minor nit: The following files in the tarballs contains french date 
lines probably due to exporting them from svn with your local locale:

  - ROADMAP
  - VERSIONING
  - maybe others?
  Maybe already fixed, I saw some conversation about in the future
  using a default locale in scripts.

Detailed report:

- Sigs and hashes OK
- contents of tarballs identical
- contents of tag and tarballs identical
  except for expected deltas

Built on

- Solaris 10 Sparc as 32 Bit Binaries
- SLES 11+12+15 (64 Bits)
- RHEL 6+7+8 (64 Bits)

For all platforms built

- with default (shared) and static modules
- with module set reallyall
- using --enable-load-all-modules
- against
  - bundled APR/APU from deps tarball
  - external APR/APU 1.7.0/1.6.1 (expat)
  - APR/APU 1.6.5/1.6.1 (expat)
  - APR/APU 1.7.x r1889104/1.7.x r1889948 (expat)
  - APR/APU 1.7.x r1889104/1.7.x r1889948 (libxml2)
  - APR/APU 1.6.x r1876940/1.6.x r1889948 (expat)

- using external libraries
  - expat 2.3.0
  - pcre 8.44
  - lua 5.4.3 (compiled with LUA_COMPAT_MODULE)
  - libxml2 2.9.12
  - libnghttp2 1.43.0
  - brotli 1.0.9
  - curl 7.76.1
  - jansson 2.13.1
  - libldap 2.4.58 (resp. 2.4.52 when using OpenSSL 0.9.8)
and
  - openssl 0.9.8zh, 1.0.2, 1.0.2u, 1.1.1, 1.1.1k, 3.0.0alpha16

- Tool chain:
- platform gcc except on Solaris
  (gcc 9.3.0 Solaris 10)
- CFLAGS: -O2 -g -Wall -fno-strict-aliasing
  - on Solaris additionally -mpcu=v9, -D_XOPEN_SOURCE,
-D_XOPEN_SOURCE_EXTENDED=1, -D__EXTENSIONS__
and -D_XPG6

All of the 884 builds succeeded.

- compiler warnings: see earlier separate mail


Tested for

- SLES 11+12 done
- SLES 15 and RHEL 6+7+8 mostly done
- Solaris 10 Sparc about 12% done
- MPMs prefork, worker, event
- default and static module builds
- log level trace8
- module set reallyall (129 modules plus 3 MPMs)
- Perl client bundle build against OpenSSL 1.1.1g plus patches, 1.1.0l, 
1.0.2u and 0.9.8zh

- OpenSSL once linked statically and once as a shared library

Every OpenSSL version in the client is tested with every OpenSSL version 
in the server.


The total number of test suite runs until now is ~8000 (more still to 
come, especially most of the Solaris ones and some of those with 
statically linked OpenSSL in combination with statically linked server 
on Linux).


Some local adjustments to tests were used:

- t/modules/buffer.t: removing huge buffer tests
  -my $bigsize = 10;
  +my $bigsize = 1;

- fixing limitrequestline overwrite which does not yet really work in 
Apache-Test/lib/Apache/TestConfig.pm

87d86
 'global LimitRequestLine setting (default is 128)',
96a96
> #   limitrequestline => 'global LimitRequestLine setting (default is 
128)',

372,373c372,373
< $vars->{limitrequestline} ||= 128;
< $vars->{limitrequestlinex2} = 2 * $vars->{limitrequestline};
---
> #$vars->{limitrequestline} ||= 128;
> #$vars->{limitrequestlinex2} = 2 * $vars->{limitrequestline};

- the temporary workaround for OpenSSL 3 when using "openssl crl -hash" 
with STDIN in Apache-Test/lib/Apache/TestSSLCA.pm is no longer 
necessary, problem fixed in OpenSSL 3.0.0alpha16



The following test failures were seen:

a A single crash in SLES 11 APU 1.6.1 APR 1.7.0 OpenSSL 1.0.2u
  build statically using event.
  Crash during ssl_io_filter_handshake calling ... CRYPTO_malloc and
  finally "glibc detected *** /path/to/bin/httpd: malloc():
  memory corruption: 0x00faafe1"
  gdb data see below.

b Two crashes on Solaris 10 Sparc,
  once APU 1.6.1 APR 1.6.5 OpenSSL 3.0.0alpha16
  and once APU 1.6.1 APR 1.7.0 OpenSSL 1.1.1k
  build dynamically both using prefork.
  Crash during shutdown when apr_pool_destroy() for the
  pchild pool calls allocator_free() (invalid node->next).
  Might be fixed in svn heads. I will check, whether the ongoing
  Solaris 

Re: Build warnings in 2.4.48

2021-05-20 Thread Rainer Jung

Am 20.05.2021 um 14:07 schrieb Noel Butler:

On 20/05/2021 21:19, Rainer Jung wrote:


Hi there,

I saw the following build warnings in 2.4.48:

modules/md/md_crypt.c:1382: warning: passing argument 1 of 
‘BIO_new_mem_buf’ discards qualifiers from pointer target type

[-Wdiscarded-qualifiers]

=> happens only when building against OpenSSL 1.0.2 (initial release, 
no letter suffix).

But in openssl 1.0.2, was that not EOL'd back in late 2019?


I think until now httpd 2.4.x still officially supports being build with 
OpenSSL starting at 0.9.8.


Best Regards,

Rainer


Build warnings in 2.4.48

2021-05-20 Thread Rainer Jung

Hi there,

I saw the following build warnings in 2.4.48:

modules/md/md_crypt.c:1382: warning: passing argument 1 of 
‘BIO_new_mem_buf’ discards qualifiers from pointer target type

[-Wdiscarded-qualifiers]

=> happens only when building against OpenSSL 1.0.2 (initial release, no 
letter suffix).



modules/ssl/ssl_engine_kernel.c:2318: warning: ‘set_challenge_creds’ 
defined but not used

On SLES 11 line number is 2321 instead of 2318

=> happens only when building against 0.9.8zh-1 (other 0.9.8 versions 
not tested)



modules/ssl/ssl_util_ssl.c:531: warning: passing argument 1 of 
‘BIO_new_mem_buf’ discards qualifiers from pointer target type

[-Wdiscarded-qualifiers]

The same again on line 544.

=> happens only when building against OpenSSL 1.0.2 (initial release, no 
letter suffix) and when building against 0.9.8zh-1 (other 0.9.8 versions 
not tested)



srclib/apr/locks/unix/proc_mutex.c:979:49: warning: 
'mutex_proc_pthread_cond_methods' defined but not used 
[-Wunused-const-variable=]


=> only on Solaris 10 Sparc (GCC 9.3.0) plus APR 1.7.x
the declaration checks for APR_USE_PROC_PTHREAD_MUTEX_COND, which is 
defined on Solaris, the use in line 1437 also checks 
HAVE_PTHREAD_MUTEX_ROBUST, which is not defined on Solaris.



Best regards,

Rainer


Re: [VOTE] Release httpd-2.4.47

2021-04-27 Thread Rainer Jung

Am 22.04.2021 um 11:25 schrieb Christophe JAILLET:

Hi, all;
   Please find below the proposed release tarball and signatures:
https://dist.apache.org/repos/dist/dev/httpd/

I would like to call a VOTE over the next few days to release this 
candidate tarball as 2.4.47:

[ ] +1: It's not just good, it's good enough!
[ ] +0: Let's have a talk.
[ ] -1: There's trouble in paradise. Here's what's wrong.

The computed digests of the tarball up for vote are:
sha1: f4281be0bf08489a51d818b596a92bfcfbb2c708 *httpd-2.4.47.tar.gz
sha256: 567d5ac72ea643e3828e8e54f32e06f1fad10095d33ae4071eeaec3c78b70a34 
*httpd-2.4.47.tar.gz
sha512: 
de4c80e1ddebe3286c234179fd01d4917f479f75a7fe958032c19a8f22546e95f31e3b50073844d09f20f54894e7d511bcd9fd2f1cd2b2c71b3a182d6e62bab3 
*httpd-2.4.47.tar.gz


The SVN tag is '2.4.47' at r1889091.

--
Christophe JAILLET



+1 to release and thanks a bunch for stepping in as RM!

I think I wasn't able to import your key from a pgp key server, but 
maybe I wasn't using the right ones. Your key is in the KEYS file, 
probably just not in the network of key servers.


Summary: all OK except for

- one single crash on SLES 11 using prefork.

Detailed report:

- Sigs and hashes OK
- contents of tarballs identical
- contents of tag and tarballs identical
  except for expected deltas

Built on

- Solaris 10 Sparc as 32 Bit Binaries
- SLES 11+12+15 (64 Bits)
- RHEL 6+7+8 (64 Bits)

For all platforms built

- with default (shared) and static modules
- with module set reallyall
- using --enable-load-all-modules
- against
  - bundled APR/APU from deps tarball
  - external APR/APU 1.7.0/1.6.1 (expat)
  - APR/APU 1.6.5/1.6.1 (expat)
  - APR/APU 1.7.x r1889104/1.7.x r1884108 (expat)
  - APR/APU 1.7.x r1889104/1.7.x r1884108 (libxml2)
  - APR/APU 1.6.x r1876940/1.6.x r1876943 (expat)
  - APR/APU 1.6.x r1876940/1.6.x r1876943 (libxml2)

- using external libraries
  - expat 2.3.0
  - pcre 8.44
  - lua 5.4.3 (compiled with LUA_COMPAT_MODULE)
  - libxml2 2.9.10
  - libnghttp2 1.43.0
  - brotli 1.0.9
  - curl 7.76.1
  - jansson 2.13.1
  - libldap 2.4.58
and
  - openssl 0.9.8zh, 1.0.2, 1.0.2u, 1.1.1, 1.1.1k, 3.0.0alpha15

- Tool chain:
- platform gcc except on Solaris
  (gcc 9.3.0 Solaris 10)
- CFLAGS: -O2 -g -Wall -fno-strict-aliasing
  - on Solaris additionally -mpcu=v9, -D_XOPEN_SOURCE,
-D_XOPEN_SOURCE_EXTENDED=1, -D__EXTENSIONS__
and -D_XPG6

All of the 852 builds done until now succeeded, 32 builds on Solaris yet 
to come.


- compiler warnings:

  - only on Solaris 10 Sparc (GCC 9.3.0): APR 1.7.x
srclib/apr/locks/unix/proc_mutex.c:979:49: warning: 
'mutex_proc_pthread_cond_methods' defined but not used 
[-Wunused-const-variable=]
=> the declaration checks for APR_USE_PROC_PTHREAD_MUTEX_COND, 
which is defined on Solaris, the use in line 1437 also checks 
HAVE_PTHREAD_MUTEX_ROBUST, which is not defined on Solaris.


  - deprecation warnings when building against OpenSSL 3.0.0, see other 
mail



Tested for

- SLES 11+12+15, RHEL 6+7+8
  - Tests for Solaris 10 Sparc still to come, when builds finish there
- MPMs prefork, worker, event
- default and static module builds
- log level trace8
- module set reallyall (129 modules plus 3 MPMs)
- Perl client bundle build against OpenSSL 1.1.1g plus patches, 1.1.0l, 
1.0.2u and 0.9.8zh

- OpenSSL once linked statically and once as a shared library

Every OpenSSL version in the client is tested with every OpenSSL version 
in the server.


The total number of test suite runs until now is ~3800 (many more still 
to come, especially those with statically linked OpenSSL and most of the 
Solaris ones).


Some local adjustments to tests were used:

- t/modules/buffer.t: removing huge buffer tests
  -my $bigsize = 10;
  +my $bigsize = 1;

- fixing limitrequestline overwrite which does not yet really work in 
Apache-Test/lib/Apache/TestConfig.pm

87d86
 'global LimitRequestLine setting (default is 128)',
96a96
> #   limitrequestline => 'global LimitRequestLine setting (default is 
128)',

372,373c372,373
< $vars->{limitrequestline} ||= 128;
< $vars->{limitrequestlinex2} = 2 * $vars->{limitrequestline};
---
> #$vars->{limitrequestline} ||= 128;
> #$vars->{limitrequestlinex2} = 2 * $vars->{limitrequestline};

- temporary workaround for a OpenSSL 3 when using "openssl crl -hash" 
with STDIN in Apache-Test/lib/Apache/TestSSLCA.pm

39a40
> my $openssl_workaround = $ENV{APACHE_TEST_OPENSSL_CMD_WORKAROUND} || 
$openssl;

426c427
< chomp(my $hash = `$openssl $type -noout -hash < $file`);
---
> chomp(my $hash = `$openssl_workaround $type -noout -hash < 
$file`);


This enables to use OpenSSL 3 everywhere in test suite configuration and 
setup and override it only in the CRL hash command line, for which it is 
currently buggy.


The following test failures were seen:

a All https tests fail between OpenSSL 0.9.8zh and 3.0.0alpha15
  Not a regression.
  Probably need to figure out how 

Re: httpd 2.4.x TLS proxy and OpenSSL 3.0.0alpha15: CRL problem

2021-04-26 Thread Rainer Jung
OK, it is a bug or behavior change in the "openssl crl" command line for 
3.0.0 that leads to the CRL symlinks not being created any more.


I raised

https://github.com/openssl/openssl/issues/15031

Regards,

Rainer

Am 26.04.2021 um 15:34 schrieb Rainer Jung:
I am still investigating further, bit this is due to the fact, that the 
CRL symlink in the crl directory is missing. Might be a local 
integration issue. Will look further.


Am 26.04.2021 um 13:27 schrieb Rainer Jung:
When building 2.4.47 using OpenSSL 3.0.0alpha15 and running the test 
suite, the proxy TLS connection fails with "Certificate Verification: 
Error (3): unable to get certificate CRL":


[Mon Apr 26 10:00:50.352111 2021] [ssl:trace3] [pid 16699:tid 
140438686086912] ssl_engine_kernel.c(2213): [remote 127.0.0.1:8532] 
OpenSSL: Loop: TLSv1.3 read encrypted extensions
[Mon Apr 26 10:00:50.352449 2021] [ssl:debug] [pid 16699:tid 
140438686086912] ssl_engine_kernel.c(1762): [remote 127.0.0.1:8532] 
AH02275: Certificate Verification, depth 0, CRL checking mode: chain 
(2) [subject: 
emailAddress=test-...@httpd.apache.org,CN=localhost,OU=httpd-test/rsa-test,O=ASF,L=San 
Francisco,ST=California,C=US / issuer: 
emailAddress=test-...@httpd.apache.org,CN=ca,OU=httpd-test,O=ASF,L=San 
Francisco,ST=California,C=US / serial: 09 / notbefore: Apr 26 07:56:58 
2021 GMT / notafter: Apr 26 07:56:58 2022 GMT]
[Mon Apr 26 10:00:50.352487 2021] [ssl:info] [pid 16699:tid 
140438686086912] [remote 127.0.0.1:8532] AH02276: Certificate 
Verification: Error (3): unable to get certificate CRL [subject: 
emailAddress=test-...@httpd.apache.org,CN=localhost,OU=httpd-test/rsa-test,O=ASF,L=San 
Francisco,ST=California,C=US / issuer: 
emailAddress=test-...@httpd.apache.org,CN=ca,OU=httpd-test,O=ASF,L=San 
Francisco,ST=California,C=US / serial: 09 / notbefore: Apr 26 07:56:58 
2021 GMT / notafter: Apr 26 07:56:58 2022 GMT]
[Mon Apr 26 10:00:50.352563 2021] [ssl:trace3] [pid 16699:tid 
140438686086912] ssl_engine_kernel.c(2223): [remote 127.0.0.1:8532] 
OpenSSL: Write: error
[Mon Apr 26 10:00:50.352567 2021] [ssl:trace3] [pid 16699:tid 
140438686086912] ssl_engine_kernel.c(2242): [remote 127.0.0.1:8532] 
OpenSSL: Exit: error in error
[Mon Apr 26 10:00:50.352570 2021] [ssl:info] [pid 16699:tid 
140438686086912] [remote 127.0.0.1:8532] AH02003: SSL Proxy connect 
failed
[Mon Apr 26 10:00:50.352597 2021] [ssl:info] [pid 16699:tid 
140438686086912] SSL Library Error: error:0A86:SSL 
routines::certificate verify failed



More complete log output:

[Mon Apr 26 10:00:50.344676 2021] [proxy:trace2] [pid 16699:tid 
140438686086912] proxy_util.c(3153): https: fam 2 socket created to 
connect to localhost
[Mon Apr 26 10:00:50.344710 2021] [proxy:debug] [pid 16699:tid 
140438686086912] proxy_util.c(3187): AH02824: https: connection 
established with 127.0.0.1:8532 (localhost)
[Mon Apr 26 10:00:50.344719 2021] [example_hooks:notice] [pid 
16699:tid 140438686086912] x_create_connection()
[Mon Apr 26 10:00:50.344726 2021] [proxy:trace1] [pid 16699:tid 
140438686086912] proxy_util.c(3359): [remote 127.0.0.1:8532] https: 
set SNI to localhost for (localhost)
[Mon Apr 26 10:00:50.344729 2021] [proxy:debug] [pid 16699:tid 
140438686086912] proxy_util.c(3371): AH00962: https: connection 
complete to [::1]:8532 (localhost)
[Mon Apr 26 10:00:50.344738 2021] [ssl:info] [pid 16699:tid 
140438686086912] [remote 127.0.0.1:8532] AH01964: Connection to child 
0 established (server localhost:8561)
[Mon Apr 26 10:00:50.344822 2021] [ssl:trace2] [pid 16699:tid 
140438686086912] ssl_engine_rand.c(126): Proxy: Seeding PRNG with 144 
bytes of entropy
[Mon Apr 26 10:00:50.344875 2021] [ssl:trace3] [pid 16699:tid 
140438686086912] ssl_engine_io.c(1242): [remote 127.0.0.1:8532] SNI 
extension for SSL Proxy request set to 'localhost'
[Mon Apr 26 10:00:50.344882 2021] [ssl:trace3] [pid 16699:tid 
140438686086912] ssl_engine_kernel.c(2204): [remote 127.0.0.1:8532] 
OpenSSL: Handshake: start
[Mon Apr 26 10:00:50.344921 2021] [ssl:trace3] [pid 16699:tid 
140438686086912] ssl_engine_kernel.c(2213): [remote 127.0.0.1:8532] 
OpenSSL: Loop: before SSL initialization
[Mon Apr 26 10:00:50.345273 2021] [example_hooks:notice] [pid 
16699:tid 140438669301504] x_create_connection()
[Mon Apr 26 10:00:50.345308 2021] [ssl:info] [pid 16699:tid 
140438669301504] [client 127.0.0.1:50940] AH01964: Connection to child 
210 established (server localhost:8532)
[Mon Apr 26 10:00:50.345356 2021] [ssl:trace3] [pid 16699:tid 
140438686086912] ssl_engine_kernel.c(2213): [remote 127.0.0.1:8532] 
OpenSSL: Loop: SSLv3/TLS write client hello
[Mon Apr 26 10:00:50.345400 2021] [ssl:trace2] [pid 16699:tid 
140438669301504] ssl_engine_rand.c(126): Server: Seeding PRNG with 144 
bytes of entropy
[Mon Apr 26 10:00:50.345459 2021] [ssl:trace3] [pid 16699:tid 
140438669301504] ssl_engine_kernel.c(2204): [client 127.0.0.1:50940] 
OpenSSL: Handshake: start
[Mon Apr 26 10:00:50.345479 2021] [ssl:trace3] [pid 16699:

Re: httpd 2.4.x TLS proxy and OpenSSL 3.0.0alpha15: CRL problem

2021-04-26 Thread Rainer Jung
I am still investigating further, bit this is due to the fact, that the 
CRL symlink in the crl directory is missing. Might be a local 
integration issue. Will look further.


Am 26.04.2021 um 13:27 schrieb Rainer Jung:
When building 2.4.47 using OpenSSL 3.0.0alpha15 and running the test 
suite, the proxy TLS connection fails with "Certificate Verification: 
Error (3): unable to get certificate CRL":


[Mon Apr 26 10:00:50.352111 2021] [ssl:trace3] [pid 16699:tid 
140438686086912] ssl_engine_kernel.c(2213): [remote 127.0.0.1:8532] 
OpenSSL: Loop: TLSv1.3 read encrypted extensions
[Mon Apr 26 10:00:50.352449 2021] [ssl:debug] [pid 16699:tid 
140438686086912] ssl_engine_kernel.c(1762): [remote 127.0.0.1:8532] 
AH02275: Certificate Verification, depth 0, CRL checking mode: chain (2) 
[subject: 
emailAddress=test-...@httpd.apache.org,CN=localhost,OU=httpd-test/rsa-test,O=ASF,L=San 
Francisco,ST=California,C=US / issuer: 
emailAddress=test-...@httpd.apache.org,CN=ca,OU=httpd-test,O=ASF,L=San 
Francisco,ST=California,C=US / serial: 09 / notbefore: Apr 26 07:56:58 
2021 GMT / notafter: Apr 26 07:56:58 2022 GMT]
[Mon Apr 26 10:00:50.352487 2021] [ssl:info] [pid 16699:tid 
140438686086912] [remote 127.0.0.1:8532] AH02276: Certificate 
Verification: Error (3): unable to get certificate CRL [subject: 
emailAddress=test-...@httpd.apache.org,CN=localhost,OU=httpd-test/rsa-test,O=ASF,L=San 
Francisco,ST=California,C=US / issuer: 
emailAddress=test-...@httpd.apache.org,CN=ca,OU=httpd-test,O=ASF,L=San 
Francisco,ST=California,C=US / serial: 09 / notbefore: Apr 26 07:56:58 
2021 GMT / notafter: Apr 26 07:56:58 2022 GMT]
[Mon Apr 26 10:00:50.352563 2021] [ssl:trace3] [pid 16699:tid 
140438686086912] ssl_engine_kernel.c(2223): [remote 127.0.0.1:8532] 
OpenSSL: Write: error
[Mon Apr 26 10:00:50.352567 2021] [ssl:trace3] [pid 16699:tid 
140438686086912] ssl_engine_kernel.c(2242): [remote 127.0.0.1:8532] 
OpenSSL: Exit: error in error
[Mon Apr 26 10:00:50.352570 2021] [ssl:info] [pid 16699:tid 
140438686086912] [remote 127.0.0.1:8532] AH02003: SSL Proxy connect failed
[Mon Apr 26 10:00:50.352597 2021] [ssl:info] [pid 16699:tid 
140438686086912] SSL Library Error: error:0A86:SSL 
routines::certificate verify failed



More complete log output:

[Mon Apr 26 10:00:50.344676 2021] [proxy:trace2] [pid 16699:tid 
140438686086912] proxy_util.c(3153): https: fam 2 socket created to 
connect to localhost
[Mon Apr 26 10:00:50.344710 2021] [proxy:debug] [pid 16699:tid 
140438686086912] proxy_util.c(3187): AH02824: https: connection 
established with 127.0.0.1:8532 (localhost)
[Mon Apr 26 10:00:50.344719 2021] [example_hooks:notice] [pid 16699:tid 
140438686086912] x_create_connection()
[Mon Apr 26 10:00:50.344726 2021] [proxy:trace1] [pid 16699:tid 
140438686086912] proxy_util.c(3359): [remote 127.0.0.1:8532] https: set 
SNI to localhost for (localhost)
[Mon Apr 26 10:00:50.344729 2021] [proxy:debug] [pid 16699:tid 
140438686086912] proxy_util.c(3371): AH00962: https: connection complete 
to [::1]:8532 (localhost)
[Mon Apr 26 10:00:50.344738 2021] [ssl:info] [pid 16699:tid 
140438686086912] [remote 127.0.0.1:8532] AH01964: Connection to child 0 
established (server localhost:8561)
[Mon Apr 26 10:00:50.344822 2021] [ssl:trace2] [pid 16699:tid 
140438686086912] ssl_engine_rand.c(126): Proxy: Seeding PRNG with 144 
bytes of entropy
[Mon Apr 26 10:00:50.344875 2021] [ssl:trace3] [pid 16699:tid 
140438686086912] ssl_engine_io.c(1242): [remote 127.0.0.1:8532] SNI 
extension for SSL Proxy request set to 'localhost'
[Mon Apr 26 10:00:50.344882 2021] [ssl:trace3] [pid 16699:tid 
140438686086912] ssl_engine_kernel.c(2204): [remote 127.0.0.1:8532] 
OpenSSL: Handshake: start
[Mon Apr 26 10:00:50.344921 2021] [ssl:trace3] [pid 16699:tid 
140438686086912] ssl_engine_kernel.c(2213): [remote 127.0.0.1:8532] 
OpenSSL: Loop: before SSL initialization
[Mon Apr 26 10:00:50.345273 2021] [example_hooks:notice] [pid 16699:tid 
140438669301504] x_create_connection()
[Mon Apr 26 10:00:50.345308 2021] [ssl:info] [pid 16699:tid 
140438669301504] [client 127.0.0.1:50940] AH01964: Connection to child 
210 established (server localhost:8532)
[Mon Apr 26 10:00:50.345356 2021] [ssl:trace3] [pid 16699:tid 
140438686086912] ssl_engine_kernel.c(2213): [remote 127.0.0.1:8532] 
OpenSSL: Loop: SSLv3/TLS write client hello
[Mon Apr 26 10:00:50.345400 2021] [ssl:trace2] [pid 16699:tid 
140438669301504] ssl_engine_rand.c(126): Server: Seeding PRNG with 144 
bytes of entropy
[Mon Apr 26 10:00:50.345459 2021] [ssl:trace3] [pid 16699:tid 
140438669301504] ssl_engine_kernel.c(2204): [client 127.0.0.1:50940] 
OpenSSL: Handshake: start
[Mon Apr 26 10:00:50.345479 2021] [ssl:trace3] [pid 16699:tid 
140438669301504] ssl_engine_kernel.c(2213): [client 127.0.0.1:50940] 
OpenSSL: Loop: before SSL initialization
[Mon Apr 26 10:00:50.345680 2021] [ssl:trace3] [pid 16699:tid 
140438669301504] ssl_engine_kernel.c(2213): [client 127.0.0.1:50940] 
OpenSSL: Loop: before SSL init

httpd 2.4.x TLS proxy and OpenSSL 3.0.0alpha15: CRL problem

2021-04-26 Thread Rainer Jung
When building 2.4.47 using OpenSSL 3.0.0alpha15 and running the test 
suite, the proxy TLS connection fails with "Certificate Verification: 
Error (3): unable to get certificate CRL":


[Mon Apr 26 10:00:50.352111 2021] [ssl:trace3] [pid 16699:tid 
140438686086912] ssl_engine_kernel.c(2213): [remote 127.0.0.1:8532] 
OpenSSL: Loop: TLSv1.3 read encrypted extensions
[Mon Apr 26 10:00:50.352449 2021] [ssl:debug] [pid 16699:tid 
140438686086912] ssl_engine_kernel.c(1762): [remote 127.0.0.1:8532] 
AH02275: Certificate Verification, depth 0, CRL checking mode: chain (2) 
[subject: 
emailAddress=test-...@httpd.apache.org,CN=localhost,OU=httpd-test/rsa-test,O=ASF,L=San 
Francisco,ST=California,C=US / issuer: 
emailAddress=test-...@httpd.apache.org,CN=ca,OU=httpd-test,O=ASF,L=San 
Francisco,ST=California,C=US / serial: 09 / notbefore: Apr 26 07:56:58 
2021 GMT / notafter: Apr 26 07:56:58 2022 GMT]
[Mon Apr 26 10:00:50.352487 2021] [ssl:info] [pid 16699:tid 
140438686086912] [remote 127.0.0.1:8532] AH02276: Certificate 
Verification: Error (3): unable to get certificate CRL [subject: 
emailAddress=test-...@httpd.apache.org,CN=localhost,OU=httpd-test/rsa-test,O=ASF,L=San 
Francisco,ST=California,C=US / issuer: 
emailAddress=test-...@httpd.apache.org,CN=ca,OU=httpd-test,O=ASF,L=San 
Francisco,ST=California,C=US / serial: 09 / notbefore: Apr 26 07:56:58 
2021 GMT / notafter: Apr 26 07:56:58 2022 GMT]
[Mon Apr 26 10:00:50.352563 2021] [ssl:trace3] [pid 16699:tid 
140438686086912] ssl_engine_kernel.c(2223): [remote 127.0.0.1:8532] 
OpenSSL: Write: error
[Mon Apr 26 10:00:50.352567 2021] [ssl:trace3] [pid 16699:tid 
140438686086912] ssl_engine_kernel.c(2242): [remote 127.0.0.1:8532] 
OpenSSL: Exit: error in error
[Mon Apr 26 10:00:50.352570 2021] [ssl:info] [pid 16699:tid 
140438686086912] [remote 127.0.0.1:8532] AH02003: SSL Proxy connect failed
[Mon Apr 26 10:00:50.352597 2021] [ssl:info] [pid 16699:tid 
140438686086912] SSL Library Error: error:0A86:SSL 
routines::certificate verify failed



More complete log output:

[Mon Apr 26 10:00:50.344676 2021] [proxy:trace2] [pid 16699:tid 
140438686086912] proxy_util.c(3153): https: fam 2 socket created to 
connect to localhost
[Mon Apr 26 10:00:50.344710 2021] [proxy:debug] [pid 16699:tid 
140438686086912] proxy_util.c(3187): AH02824: https: connection 
established with 127.0.0.1:8532 (localhost)
[Mon Apr 26 10:00:50.344719 2021] [example_hooks:notice] [pid 16699:tid 
140438686086912] x_create_connection()
[Mon Apr 26 10:00:50.344726 2021] [proxy:trace1] [pid 16699:tid 
140438686086912] proxy_util.c(3359): [remote 127.0.0.1:8532] https: set 
SNI to localhost for (localhost)
[Mon Apr 26 10:00:50.344729 2021] [proxy:debug] [pid 16699:tid 
140438686086912] proxy_util.c(3371): AH00962: https: connection complete 
to [::1]:8532 (localhost)
[Mon Apr 26 10:00:50.344738 2021] [ssl:info] [pid 16699:tid 
140438686086912] [remote 127.0.0.1:8532] AH01964: Connection to child 0 
established (server localhost:8561)
[Mon Apr 26 10:00:50.344822 2021] [ssl:trace2] [pid 16699:tid 
140438686086912] ssl_engine_rand.c(126): Proxy: Seeding PRNG with 144 
bytes of entropy
[Mon Apr 26 10:00:50.344875 2021] [ssl:trace3] [pid 16699:tid 
140438686086912] ssl_engine_io.c(1242): [remote 127.0.0.1:8532] SNI 
extension for SSL Proxy request set to 'localhost'
[Mon Apr 26 10:00:50.344882 2021] [ssl:trace3] [pid 16699:tid 
140438686086912] ssl_engine_kernel.c(2204): [remote 127.0.0.1:8532] 
OpenSSL: Handshake: start
[Mon Apr 26 10:00:50.344921 2021] [ssl:trace3] [pid 16699:tid 
140438686086912] ssl_engine_kernel.c(2213): [remote 127.0.0.1:8532] 
OpenSSL: Loop: before SSL initialization
[Mon Apr 26 10:00:50.345273 2021] [example_hooks:notice] [pid 16699:tid 
140438669301504] x_create_connection()
[Mon Apr 26 10:00:50.345308 2021] [ssl:info] [pid 16699:tid 
140438669301504] [client 127.0.0.1:50940] AH01964: Connection to child 
210 established (server localhost:8532)
[Mon Apr 26 10:00:50.345356 2021] [ssl:trace3] [pid 16699:tid 
140438686086912] ssl_engine_kernel.c(2213): [remote 127.0.0.1:8532] 
OpenSSL: Loop: SSLv3/TLS write client hello
[Mon Apr 26 10:00:50.345400 2021] [ssl:trace2] [pid 16699:tid 
140438669301504] ssl_engine_rand.c(126): Server: Seeding PRNG with 144 
bytes of entropy
[Mon Apr 26 10:00:50.345459 2021] [ssl:trace3] [pid 16699:tid 
140438669301504] ssl_engine_kernel.c(2204): [client 127.0.0.1:50940] 
OpenSSL: Handshake: start
[Mon Apr 26 10:00:50.345479 2021] [ssl:trace3] [pid 16699:tid 
140438669301504] ssl_engine_kernel.c(2213): [client 127.0.0.1:50940] 
OpenSSL: Loop: before SSL initialization
[Mon Apr 26 10:00:50.345680 2021] [ssl:trace3] [pid 16699:tid 
140438669301504] ssl_engine_kernel.c(2213): [client 127.0.0.1:50940] 
OpenSSL: Loop: before SSL initialization
[Mon Apr 26 10:00:50.345702 2021] [ssl:debug] [pid 16699:tid 
140438669301504] ssl_engine_kernel.c(2374): [client 127.0.0.1:50940] 
AH02043: SSL virtual host for servername localhost found
[Mon Apr 26 10:00:50.345833 2021] 

OpenSSL 3.0.0 deprecations

2021-04-26 Thread Rainer Jung
FYI: here's a list of symbols for which I get deprecation warnings when 
compiling httpd 2.4.47 (plus bundled APU) against current OpenSSL 
3.0.0alpha15:


srclib/apr-util/crypto/apr_crypto_openssl.c:141:5 
ENGINE_load_builtin_engines
srclib/apr-util/crypto/apr_crypto_openssl.c:142:5 
ENGINE_register_all_complete

srclib/apr-util/crypto/apr_crypto_openssl.c:208:9 ENGINE_finish
srclib/apr-util/crypto/apr_crypto_openssl.c:209:9 ENGINE_free
srclib/apr-util/crypto/apr_crypto_openssl.c:326:9 ENGINE_by_id
srclib/apr-util/crypto/apr_crypto_openssl.c:330:9 ENGINE_init
srclib/apr-util/crypto/apr_crypto_openssl.c:331:13 ENGINE_free

modules/ssl/ssl_engine_config.c:610:5 ENGINE_by_id
modules/ssl/ssl_engine_config.c:612:9 ENGINE_free
modules/ssl/ssl_engine_config.c:617:9 ENGINE_get_first
modules/ssl/ssl_engine_config.c:619:13 ENGINE_get_id
modules/ssl/ssl_engine_config.c:620:42 ENGINE_get_name
modules/ssl/ssl_engine_config.c:623:13 ENGINE_get_next
modules/ssl/ssl_engine_init.c:102:5 DH_new
modules/ssl/ssl_engine_init.c:113:5 DH_set0_pqg
modules/ssl/ssl_engine_init.c:114:9 DH_free
modules/ssl/ssl_engine_init.c:1434:9 DH_bits
modules/ssl/ssl_engine_init.c:1437:9 DH_free
modules/ssl/ssl_engine_init.c:1447:9 EC_KEY_new_by_curve_name
modules/ssl/ssl_engine_init.c:1469:5 EC_KEY_free
modules/ssl/ssl_engine_init.c:152:9 DH_free
modules/ssl/ssl_engine_init.c:1722:9 SRP_VBASE_free
modules/ssl/ssl_engine_init.c:457:9 ENGINE_by_id
modules/ssl/ssl_engine_init.c:467:13 ENGINE_ctrl
modules/ssl/ssl_engine_init.c:471:9 ENGINE_set_default
modules/ssl/ssl_engine_init.c:482:9 ENGINE_free
modules/ssl/ssl_engine_init.c:548:9 SRP_VBASE_new
modules/ssl/ssl_engine_init.c:557:9 SRP_VBASE_init
modules/ssl/ssl_engine_init.c:565:9 SSL_CTX_set_srp_username_callback
modules/ssl/ssl_engine_init.c:567:9 SSL_CTX_set_srp_cb_arg
modules/ssl/ssl_engine_init.c:832:5 SSL_CTX_set_tmp_dh_callback
modules/ssl/ssl_engine_kernel.c:2632:9 HMAC_Init_ex
modules/ssl/ssl_engine_kernel.c:2653:9 HMAC_Init_ex
modules/ssl/ssl_engine_kernel.c:2781:5 SSL_get_srp_username
modules/ssl/ssl_engine_kernel.c:2788:9 SRP_VBASE_get1_by_user
modules/ssl/ssl_engine_kernel.c:2794:5 SSL_set_srp_server_param
modules/ssl/ssl_engine_kernel.c:2796:9 SRP_user_pwd_free
modules/ssl/ssl_engine_kernel.c:2804:5 SRP_user_pwd_free
modules/ssl/ssl_engine_kernel.c:545:5 SSL_get_srp_username
modules/ssl/ssl_engine_log.c:90:5 ERR_peek_error_line_data
modules/ssl/ssl_engine_pphrase.c:856:5 ENGINE_by_id
modules/ssl/ssl_engine_pphrase.c:864:5 ENGINE_init
modules/ssl/ssl_engine_pphrase.c:877:9 ENGINE_ctrl_cmd_string
modules/ssl/ssl_engine_pphrase.c:886:9 ENGINE_ctrl_cmd
modules/ssl/ssl_engine_pphrase.c:896:5 ENGINE_load_private_key
modules/ssl/ssl_engine_pphrase.c:904:5 ENGINE_finish
modules/ssl/ssl_engine_pphrase.c:905:5 ENGINE_free
modules/ssl/ssl_engine_vars.c:432:9 SSL_get_srp_username
modules/ssl/ssl_engine_vars.c:437:9 SSL_get_srp_userinfo
modules/ssl/ssl_util_ssl.c:474:5 PEM_read_bio_DHparams
modules/ssl/ssl_util_ssl.c:487:5 PEM_read_bio_ECPKParameters

support/ab.c:756:25 EVP_PKEY_get1_EC_KEY
support/ab.c:757:25 EC_KEY_get0_group
support/ab.c:758:25 EC_KEY_free

modules/http2/h2_push.c:477:5 SHA256_Update
modules/http2/h2_push.c:487:5 SHA256_Init
modules/http2/h2_push.c:492:5 SHA256_Final

modules/md/md_crypt.c:530:5 EVP_PKEY_get1_RSA
modules/md/md_crypt.c:535:5 RSA_get0_key
modules/md/md_crypt.c:542:5 EVP_PKEY_get1_RSA
modules/md/md_crypt.c:547:5 RSA_get0_key


Re: mod_http2: .gitignore contains Makefile.in

2020-08-10 Thread Rainer Jung
I just now saw, that the file only exists in the 2.4.x branch, not in 
trunk. Should we remove the .gitignore in 2.4.x? Or add in trunk? What 
would be better for your updates between your Github variant and the 
bundled one?


Regards,

Rainer

Am 10.08.2020 um 16:43 schrieb Stefan Eissing:




Am 10.08.2020 um 16:38 schrieb Rainer Jung :

Hi there,

the .gitignore file in modules/http2 contains "Makefile.in". That's probably OK 
for the upstream Github variant, which contains a Makefile.am, but in our svn repos for 
httpd the Makefile.in is needed as the source of the Makefile generation.

Of course our svn doesn't care about the .gitignore, but eg. I now locally keep 
the sources in git and thus lost the Makefile.in. I can live with local 
modifications, but I think it would be more correct to drop Makefile.in from 
the modules/http2/.gitignore in our svn repos.

WDYT?


Sounds like a plan.



Regards,

Rainer


mod_http2: .gitignore contains Makefile.in

2020-08-10 Thread Rainer Jung

Hi there,

the .gitignore file in modules/http2 contains "Makefile.in". That's 
probably OK for the upstream Github variant, which contains a 
Makefile.am, but in our svn repos for httpd the Makefile.in is needed as 
the source of the Makefile generation.


Of course our svn doesn't care about the .gitignore, but eg. I now 
locally keep the sources in git and thus lost the Makefile.in. I can 
live with local modifications, but I think it would be more correct to 
drop Makefile.in from the modules/http2/.gitignore in our svn repos.


WDYT?

Regards,

Rainer


Re: svn commit: r40863 - /dev/httpd/ /release/httpd/

2020-08-05 Thread Rainer Jung

Thanks for the explanation and sorry about the wrong alarm!

Best regards,

Rainer

Am 06.08.2020 um 00:31 schrieb Daniel Ruggeri:

Hi, Rainer;
Right - this file gets rewritten by the announce.sh script just before 
the notification goes out. This is done to ensure that the date is 
correct and to ensure the type of release (bug, security, enhancement) 
is correct. It appears as though the file was just changed, but really 
it's just because the text was bumped as-is from the 'dev' location to 
the 'dist' location.

--
Daniel Ruggeri

On August 5, 2020 7:23:33 AM CDT, Rainer Jung  
wrote:


Could you fix the date (September 21, 2018 sems wrong).

Thanks!

Rainer

Am 05.08.2020 um 13:32 schrieb drugg...@apache.org:

Author: druggeri
Date: Wed Aug 5 11:32:51 2020
New Revision: 40863

Log:
Push 2.4.46 up to the release directory

Added:
release/httpd/CHANGES_2.4.46
- copied unchanged from r40862, dev/httpd/CHANGES_2.4.46
release/httpd/httpd-2.4.46.tar.bz2
- copied unchanged from r40862, dev/httpd/httpd-2.4.46.tar.bz2
release/httpd/httpd-2.4.46.tar.bz2.asc
- copied unchanged from r40862, dev/httpd/httpd-2.4.46.tar.bz2.asc
release/httpd/httpd-2.4.46.tar.bz2.md5
- copied unchanged from r40862, dev/httpd/httpd-2.4.46.tar.bz2.md5
release/httpd/httpd-2.4.46.tar.bz2.sha1
- copied unchanged from r40862, dev/httpd/httpd-2.4.46.tar.bz2.sha1
release/httpd/httpd-2.4.46.tar.bz2.sha256
- copied unchanged from r40862,
dev/httpd/httpd-2.4.46.tar.bz2.sha256
release/httpd/httpd-2.4.46.tar.bz2.sha512
- copied unchanged from r40862,
dev/httpd/httpd-2.4.46.tar.bz2.sha512
release/httpd/httpd-2.4.46.tar.gz
- copied unchanged from r40862, dev/httpd/httpd-2.4.46.tar.gz
release/httpd/httpd-2.4.46.tar.gz.asc
- copied unchanged from r40862, dev/httpd/httpd-2.4.46.tar.gz.asc
release/httpd/httpd-2.4.46.tar.gz.md5
- copied unchanged from r40862, dev/httpd/httpd-2.4.46.tar.gz.md5
release/httpd/httpd-2.4.46.tar.gz.sha1
- copied unchanged from r40862, dev/httpd/httpd-2.4.46.tar.gz.sha1
release/httpd/httpd-2.4.46.tar.gz.sha256
- copied unchanged from r40862, dev/httpd/httpd-2.4.46.tar.gz.sha256
release/httpd/httpd-2.4.46.tar.gz.sha512
- copied unchanged from r40862, dev/httpd/httpd-2.4.46.tar.gz.sha512
Removed:
dev/httpd/CHANGES_2.4
dev/httpd/CHANGES_2.4.46
dev/httpd/httpd-2.4.46-deps.tar.bz2
dev/httpd/httpd-2.4.46-deps.tar.bz2.asc
dev/httpd/httpd-2.4.46-deps.tar.bz2.md5
dev/httpd/httpd-2.4.46-deps.tar.bz2.sha1
dev/httpd/httpd-2.4.46-deps.tar.bz2.sha256
dev/httpd/httpd-2.4.46-deps.tar.bz2.sha512
dev/httpd/httpd-2.4.46-deps.tar.gz
dev/httpd/httpd-2.4.46-deps.tar.gz.asc
dev/httpd/httpd-2.4.46-deps.tar.gz.md5
dev/httpd/httpd-2.4.46-deps.tar.gz.sha1
dev/httpd/httpd-2.4.46-deps.tar.gz.sha256
dev/httpd/httpd-2.4.46-deps.tar.gz.sha512
dev/httpd/httpd-2.4.46.tar.bz2
dev/httpd/httpd-2.4.46.tar.bz2.asc
dev/httpd/httpd-2.4.46.tar.bz2.md5
dev/httpd/httpd-2.4.46.tar.bz2.sha1
dev/httpd/httpd-2.4.46.tar.bz2.sha256
dev/httpd/httpd-2.4.46.tar.bz2.sha512
dev/httpd/httpd-2.4.46.tar.gz
dev/httpd/httpd-2.4.46.tar.gz.asc
dev/httpd/httpd-2.4.46.tar.gz.md5
dev/httpd/httpd-2.4.46.tar.gz.sha1
dev/httpd/httpd-2.4.46.tar.gz.sha256
dev/httpd/httpd-2.4.46.tar.gz.sha512
Modified:
release/httpd/Announcement2.4.html
release/httpd/Announcement2.4.txt
release/httpd/CHANGES_2.4

Modified: release/httpd/Announcement2.4.html

--- release/httpd/Announcement2.4.html (original)
+++ release/httpd/Announcement2.4.html Wed Aug 5 11:32:51 2020
@@ -49,27 +49,27 @@



- Apache HTTP Server 2.4.43 Released
+ Apache HTTP Server 2.4.46 Released


- April 01, 2020
+ September 21, 2018


The Apache Software Foundation and the Apache HTTP Server
Project are
pleased to https://www.apache.org/dist/httpd/Announcement2.4.html;>announce
- the release of version 2.4.43 of the Apache
+ the release of version 2.4.46 of the Apache
HTTP Server ("Apache"). This version of Apache is our latest GA
release of the new generation 2.4.x branch of Apache HTTPD and
represents fifteen years of innovation by the project, and is
recommended over all previous releases. This release of Apache is
- a security, feature and bug fix release.
+ a feature and bug fix release.


  

Re: svn commit: r40863 - /dev/httpd/ /release/httpd/

2020-08-05 Thread Rainer Jung
11:32:51 2020
@@ -1,19 +1,19 @@
-Apache HTTP Server 2.4.43 Released
+Apache HTTP Server 2.4.46 Released
  
-   April 01, 2020

+   September 21, 2018
  
 The Apache Software Foundation and the Apache HTTP Server Project

-   are pleased to announce the release of version 2.4.43 of the Apache
+   are pleased to announce the release of version 2.4.46 of the Apache
 HTTP Server ("Apache").  This version of Apache is our latest GA
 release of the new generation 2.4.x branch of Apache HTTPD and
 represents fifteen years of innovation by the project, and is
 recommended over all previous releases. This release of Apache is
-   a security, feature and bug fix release.
+   a feature and bug fix release.
  
 We consider this release to be the best version of Apache available, and

 encourage users of all prior versions to upgrade.
  
-   Apache HTTP Server 2.4.43 is available for download from:

+   Apache HTTP Server 2.4.46 is available for download from:
  
   https://httpd.apache.org/download.cgi
  
@@ -24,7 +24,7 @@

   https://httpd.apache.org/docs/trunk/new_features_2_4.html
  
 Please see the CHANGES_2.4 file, linked from the download page, for a

-   full list of changes. A condensed list, CHANGES_2.4.43 includes only
+   full list of changes. A condensed list, CHANGES_2.4.46 includes only
 those changes introduced since the prior 2.4 release.  A summary of all
 of the security vulnerabilities addressed in this and earlier releases
 is available:

Modified: release/httpd/CHANGES_2.4
==
--- release/httpd/CHANGES_2.4 (original)
+++ release/httpd/CHANGES_2.4 Wed Aug  5 11:32:51 2020
@@ -1,6 +1,78 @@
   -*- coding: utf-8 -*-
+Changes with Apache 2.4.46
+  *) mod_proxy_fcgi: Fix build warnings for Windows platform
+ [Eric Covener, Christophe Jaillet]
+
+Changes with Apache 2.4.45
+
+  *) mod_http2: remove support for abandoned http-wg draft
+ <https://datatracker.ietf.org/doc/draft-kazuho-h2-cache-digest/>.
+ [Stefan Eissing]
+
+Changes with Apache 2.4.44
+
+  *) mod_proxy_uwsgi: Error out on HTTP header larger than 16K (hard
+ protocol limit).  [Yann Ylavic]
+
+  *) mod_http2:
+ Fixes <https://github.com/icing/mod_h2/issues/200>:
+ "LimitRequestFields 0" now disables the limit, as documented.
+ Fixes <https://github.com/icing/mod_h2/issues/201>:
+ Do not count repeated headers with same name against the field
+ count limit. The are merged internally, as if sent in a single HTTP/1 
line.
+ [Stefan Eissing]
+
+  *) mod_http2: Avoid segfaults in case of handling certain responses for
+ already aborted connections.  [Stefan Eissing, Ruediger Pluem]
+
+  *) mod_http2: The module now handles master/secondary connections and has 
marked
+ methods according to use. [Stefan Eissing]
+
+  *) core: Drop an invalid Last-Modified header value coming
+ from a FCGI/CGI script instead of replacing it with Unix epoch.
+ [Yann Ylavic, Luca Toscano]
+
+  *) Add support for strict content-length parsing through addition of
+ ap_parse_strict_length() [Yann Ylavic]
+
+  *) mod_proxy_fcgi: ProxyFCGISetEnvIf unsets variables when expression
+ evaluates to false.  PR64365. [Michael König ]
+
+  *) mod_proxy_http: flush spooled request body in one go to avoid
+ leaking (or long lived) temporary file. PR 64452. [Yann Ylavic]
+
+  *) mod_ssl: Fix a race condition and possible crash when using a proxy client
+ certificate (SSLProxyMachineCertificateFile).
+ [Armin Abfalterer ]
+
+  *) mod_ssl: Fix memory leak in stapling code. PR63687. [Stefan Eissing]
+
+  *) mod_http2: Fixed regression that no longer set H2_STREAM_ID and 
H2_STREAM_TAG.
+ PR64330 [Stefan Eissing]
+
+  *) mod_http2: Fixed regression that caused connections to close when 
mod_reqtimeout
+ was configured with a handshake timeout. Fixes gitub issue #196.
+ [Stefan Eissing]
+
+  *) mod_proxy_http2: the "ping" proxy parameter
+ (see <https://httpd.apache.org/docs/2.4/mod/mod_proxy.html>) is now used
+ when checking the liveliness of a new or reused h2 connection to the 
backend.
+ With short durations, this makes load-balancing more responsive. The 
module
+ will hold back requests until ping conditions are met, using features of 
the
+ HTTP/2 protocol alone. [Ruediger Pluem, Stefan Eissing]
+
+  *) core: httpd is no longer linked against -lsystemd if mod_systemd
+ is enabled (and built as a DSO).  [Rainer Jung]
+
+  *) mod_proxy_http2: respect ProxyTimeout settings on backend connections
+ while waiting on incoming data. [Ruediger Pluem, Stefan Eissing]
+
  Changes with Apache 2.4.43
  
+  *) mod_ssl: Fix memory leak of OCSP stapling response. [Yann Ylavic]

+
+Changes with Apache 2.4.42
+
*

Re: First impressions from OpenSSL 3.0.0 and httpd 2.4.45

2020-08-04 Thread Rainer Jung
Concerning the failures with OpenSSL 3.0.0 in t/ssl/proxy.t, this should 
be gone with the next alpha or beta of OpenSSL 3.0.0.


The culprit is indeed:

> [ssl:info] [pid 9162:tid 140326166714128] [remote 127.0.0.1:8532]
> AH02276: Certificate Verification: Error (3): unable to get certificate
> CRL [subject:
> 
emailAddress=test-...@httpd.apache.org,CN=localhost,OU=httpd-test/rsa-test,O=ASF,L=San 


> Francisco,ST=California,C=US / issuer:
> emailAddress=test-...@httpd.apache.org,CN=ca,OU=httpd-test,O=ASF,L=San
> Francisco,ST=California,C=US / serial: 0C / notbefore: Jul 30 23:29:05
> 2020 GMT / notafter: Jul 30 23:29:05 2021 GMT]

The reason is, that lib/Apache/TestSSLCA.pm does not use the injected 
"APACHE_TEST_OPENSSL_CMD" in one line, where it uses "`openssl ...`" 
instead of "`$openssl ...`". And this happens exactly when the hash file 
for ca-bundle.crt gets created. So instead of the older 1.1.1 openssl I 
inject during configure, the new 3.0.0 gets used to create the hash 
file. That would be fine, but OpenSSL 3.0.0 has a bug just fixed very 
recently (not yet released), that "openssl crl" can not read from STDIN. 
Which is what we do.


I'll commit the "$openssl" instead of "openssl" in backticks for 
lib/Apache/TestSSLCA.pm to make its behavior more consistent.


Concerning the failures when the test client uses OpenSSL 0.9.8 I was 
able to provide OpenSSL 3.0.0 in the server with a auto-loaded 
openssl.cnf which contained the lines to load the legacy provider. The 
provider got loaded, but still the handshakes with the old OpenSSL fail. 
Don't know why. Probably not the biggest problem, because 0.9.8 based 
clients should really not matter when thinking about 3.0.0 support in 
the server.


Regards,

Rainer

Am 01.08.2020 um 17:44 schrieb Rainer Jung:

Hi there,

during release testing for 2.4.45 I also built and tested using OpenSSL 
3.0.0alpha5 on the server. Overall first results are pretty good:


- a few deprecation warnings during compilation:

modules/ssl/ssl_engine_config.c:610:5: warning: 'ENGINE_by_id' is 
deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_config.c:612:9: warning: 'ENGINE_free' is 
deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_config.c:617:9: warning: 'ENGINE_get_first' is 
deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_config.c:619:13: warning: 'ENGINE_get_id' is 
deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_config.c:620:42: warning: 'ENGINE_get_name' is 
deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_config.c:623:13: warning: 'ENGINE_get_next' is 
deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_init.c:457:9: warning: 'ENGINE_by_id' is 
deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_init.c:467:13: warning: 'ENGINE_ctrl' is 
deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_init.c:471:9: warning: 'ENGINE_set_default' is 
deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_init.c:482:9: warning: 'ENGINE_free' is 
deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_kernel.c:2611:9: warning: 'HMAC_Init_ex' is 
deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_kernel.c:2632:9: warning: 'HMAC_Init_ex' is 
deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_log.c:90:5: warning: 'ERR_peek_error_line_data' 
is deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_pphrase.c:856:5: warning: 'ENGINE_by_id' is 
deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_pphrase.c:864:5: warning: 'ENGINE_init' is 
deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_pphrase.c:877:9: warning: 
'ENGINE_ctrl_cmd_string' is deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_pphrase.c:886:9: warning: 'ENGINE_ctrl_cmd' is 
deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_pphrase.c:896:5: warning: 
'ENGINE_load_private_key' is deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_pphrase.c:904:5: warning: 'ENGINE_finish' is 
deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_pphrase.c:905:5: warning: 'ENGINE_free' is 
deprecated [-Wdeprecated-declarations]


- a few const warnings

modules/ssl/ssl_engine_kernel.c:608:55: warning: passing argument 2 of 
'sk_SSL_CIPHER_find' discards 'const' qualifier from pointer target type 
[-Wdiscarded-qualifiers]
modules/ssl/ssl_engine_kernel.c:627:61: warning: passing argument 2 of 
'sk_SSL_CIPHER_find' discards 'const' qualifier from pointer target type 
[-Wdiscarded-qualifiers]
modules/ssl/ssl_engine_kernel.c:638:57: warning: passing argument 2 of 
'sk_SSL_CIPHER_find' discards 'const' qualifier from pointer target type 
[-Wdiscarded-qualifiers]
modules/ssl/ssl_engine_kernel.c:1039:49: warning: passing argument 2 of 
'sk_SSL_CIPHER_find' discards 'const' qualifier from pointer target type 
[-Wdiscarded-qualifiers]


and unit tests show two prob

Re: [VOTE] Release httpd-2.4.46

2020-08-04 Thread Rainer Jung

Am 01.08.2020 um 16:13 schrieb Daniel Ruggeri:

Hi, all;
    Third time is a charm! Please find below the proposed release tarball
and signatures:
https://dist.apache.org/repos/dist/dev/httpd/

I would like to call a VOTE over the next few days to release this
candidate tarball as 2.4.46:
[X] +1: It's not just good, it's good enough!
[ ] +0: Let's have a talk.
[ ] -1: There's trouble in paradise. Here's what's wrong.

The computed digests of the tarball up for vote are:
sha1: 15adb7eb3dc97e89c8a4237901a9d6887056ab98 *httpd-2.4.46.tar.gz
sha256: 44b759ce932dc090c0e75c0210b4485ebf6983466fb8ca1b446c8168e1a1aec2
*httpd-2.4.46.tar.gz
sha512:
5801c1dd0365f706a5e2365e58599b5adac674f3c66b0f39249909841e6cdf16bfdfe001fbd668f323bf7b6d14b116b5e7af49867d456336fad5e685ba020b15
*httpd-2.4.46.tar.gz

The SVN tag is '2.4.46' at r1880505.


+1 to release and thanks a bunch for multi-RM!

Summary: all OK except for

- 12 shutdown crashes on Solaris, all for prefork (already observed 
previously). Happens in mod_watchdog during server shutdown.

gdb info at end. Not a regression.

Detailed report:

- Sigs and hashes OK
- contents of tarballs identical
- contents of tag and tarballs identical
  except for expected deltas

Built and test results based on 2.4.45 but are valid for 2.4.46 due to 
the minimal code change between them.


Built on

- Solaris 10 Sparc as 32 Bit Binaries
- SLES 11+12+15 (64 Bits)
- RHEL 6+7+8 (64 Bits)

For all platforms built

- with default (shared) and static modules
- with module set reallyall
- using --enable-load-all-modules
- against external APR/APU 1.7.0/1.6.1
  plus APR/APU 1.6.5/1.6.1
  plus APR/APU 1.7.x r1880146/1.7.x r1880148 with expat
  plus APR/APU 1.7.x r1880146/1.7.x r1880148 with libxml2
  plus APR/APU from deps tarball

- using external libraries
  - expat 2.2.9
  - pcre 8.44
  - lua 5.3.5 (compiled with LUA_COMPAT_MODULE)
  - libxml2 2.9.10
  - libnghttp2 1.41.0
  - brotli 1.0.7
  - curl 7.71.1
  - jansson 2.13.1
  - libldap 2.4.50
and
  - openssl 0.9.8zh, 1.0.2, 1.0.2u, 1.0.1e, 1.0.1l, 1.1.1, 1.1.1g plus 
patches (head of master on 2020-07-11), 3.0.0alpha5


- Tool chain:
- platform gcc except on Solaris
  (gcc 9.3.0 Solaris 10)
- CFLAGS: -O2 -g -Wall -fno-strict-aliasing
  - on Solaris additionally -mpcu=v9, -D_XOPEN_SOURCE,
-D_XOPEN_SOURCE_EXTENDED=1, -D__EXTENSIONS__
and -D_XPG6

All of the 1064 builds succeeded.

- compiler warnings:

  - only on Solaris (GCC 9.3.0):
srclib/apr/locks/unix/proc_mutex.c:979:49: warning: 
'mutex_proc_pthread_cond_methods' defined but not used 
[-Wunused-const-variable=]


  - deprecation warnings when building against OpenSSL 3.0.0, see other 
thread



Tested for

- Solaris 10, SLES 11+12+15, RHEL 6+7+8
- MPMs prefork, worker, event
- default and static module builds
- log level trace8
- module set reallyall (128 modules plus 3 MPMs)
- Perl client bundle build against OpenSSL 1.1.1g plus patches, 1.1.0l, 
1.0.2u and 0.9.8zh

- OpenSSL once linked statically and once as a shared library

Every OpenSSL version in the client tested with every OpenSSL version in 
the server. Nearly all tests with dynamically linked OpenSSL are done.


The total number of test suite runs was 5913 (many more to come ...).

Some local adjustments to tests were used:

- t/modules/buffer.t: removing huge buffer tests
  -my $bigsize = 10;
  +my $bigsize = 1;

- fixing "sub which" in Apache-Test/lib/Apache/TestConfig.pm
  +# No need to search PATH components
  +# if $program already contains a path
  +return $program if !OSX and !WINFU and
  +$program =~ /\// and -f $program and -x $program;
  +

- fixing limitrequestline overwrite which does not yet really work

The following test failures were seen:

a Crashes only on Solaris, only with prefork MPM and
  dynamically linked builds.
  The crash seems to happen only at the end of a process during pchild
  clean up and it might be problematic, that the watchdog thread at that
  time still exists.
  gdb info see at end.

b Tests 2 of t/apache/pr35292.t
  Tests 2 at line 29
  Only once on Solaris. Unclear failure showing the socket
  was disconected, but the next test case sees correct response content.

c Tests 27 and 28 of t/modules/http2.t
  Tests 27 and 28 at lines 303 and 304
  Response status and content length undef for
  test case: TC0015, necho.pl 10x10:
  GET 
http://localhost:8536/modules/h2/necho.pl?count=10=0123456789

  Only once on Solaris.

d Tests 207 and 265 of t/ssl/proxy.t
  eat_post received "502 Proxy Error".
  Each of the two failed test cases only once on Solaris.

e OpenSSL 3.0.0 and t/ssl/proxy.t
  eat_post fails always, see other thread about OpenSSL 3.0.0

f All https tests fail between OpenSSL 0.9.8zh and 3.0.0alpha5
  Probably need to figure out how to load the legacy provider
  during the tests

g Test 5 in t/modules/dav.t line 69:
  Not a regression.
  Only once on SLES 11.
  Creation, modified and now times not in 

First impressions from OpenSSL 3.0.0 and httpd 2.4.45

2020-08-01 Thread Rainer Jung

Hi there,

during release testing for 2.4.45 I also built and tested using OpenSSL 
3.0.0alpha5 on the server. Overall first results are pretty good:


- a few deprecation warnings during compilation:

modules/ssl/ssl_engine_config.c:610:5: warning: 'ENGINE_by_id' is 
deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_config.c:612:9: warning: 'ENGINE_free' is 
deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_config.c:617:9: warning: 'ENGINE_get_first' is 
deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_config.c:619:13: warning: 'ENGINE_get_id' is 
deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_config.c:620:42: warning: 'ENGINE_get_name' is 
deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_config.c:623:13: warning: 'ENGINE_get_next' is 
deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_init.c:457:9: warning: 'ENGINE_by_id' is 
deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_init.c:467:13: warning: 'ENGINE_ctrl' is 
deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_init.c:471:9: warning: 'ENGINE_set_default' is 
deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_init.c:482:9: warning: 'ENGINE_free' is 
deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_kernel.c:2611:9: warning: 'HMAC_Init_ex' is 
deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_kernel.c:2632:9: warning: 'HMAC_Init_ex' is 
deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_log.c:90:5: warning: 'ERR_peek_error_line_data' 
is deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_pphrase.c:856:5: warning: 'ENGINE_by_id' is 
deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_pphrase.c:864:5: warning: 'ENGINE_init' is 
deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_pphrase.c:877:9: warning: 
'ENGINE_ctrl_cmd_string' is deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_pphrase.c:886:9: warning: 'ENGINE_ctrl_cmd' is 
deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_pphrase.c:896:5: warning: 
'ENGINE_load_private_key' is deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_pphrase.c:904:5: warning: 'ENGINE_finish' is 
deprecated [-Wdeprecated-declarations]
modules/ssl/ssl_engine_pphrase.c:905:5: warning: 'ENGINE_free' is 
deprecated [-Wdeprecated-declarations]


- a few const warnings

modules/ssl/ssl_engine_kernel.c:608:55: warning: passing argument 2 of 
'sk_SSL_CIPHER_find' discards 'const' qualifier from pointer target type 
[-Wdiscarded-qualifiers]
modules/ssl/ssl_engine_kernel.c:627:61: warning: passing argument 2 of 
'sk_SSL_CIPHER_find' discards 'const' qualifier from pointer target type 
[-Wdiscarded-qualifiers]
modules/ssl/ssl_engine_kernel.c:638:57: warning: passing argument 2 of 
'sk_SSL_CIPHER_find' discards 'const' qualifier from pointer target type 
[-Wdiscarded-qualifiers]
modules/ssl/ssl_engine_kernel.c:1039:49: warning: passing argument 2 of 
'sk_SSL_CIPHER_find' discards 'const' qualifier from pointer target type 
[-Wdiscarded-qualifiers]


and unit tests show two problems, one will be fixed in OpenSSL itself:

- during unit test preparation, our test script create a PKCS12 store 
with default encoding params. That's known to be broken in alpha5. So 
the "-configure" step of "t/TEST" should be run before the actual 
testing with a stable version of OpenSSL.

https://github.com/openssl/openssl/pull/12540
https://github.com/openssl/openssl/issues/11672

- independent of OpenSSL 3.0.0: to work around the previous observation 
I tried using the env var "APACHE_TEST_OPENSSL_CMD". Unfortunately this 
is slightly broken, because it tests for the existence using the "which" 
function in TestConfig.pm and that function is broken when used for a 
command containing a path component. I temporarily fixed it using:


@@ -1782,6 +1782,11 @@

 return undef unless $program;

+# No need to search PATH components
+# if $program already contains a path
+return $program if !OSX and !WINFU and
+$program =~ /\// and -f $program and -x $program;
+
 my @dirs = File::Spec->path();

 require Config;


- when testing with client >= OpenSSL 1.1.0 against 3.0.0alpha5, only 
t/ssl/proxy.t shows failures, especially in eat_post but already during 
TLS handshake:


[ssl:info] [pid 9162:tid 140326149928720] [client 127.0.0.1:56312] 
AH01964: Connection to child 82 established (server localhost:8532)


[ssl:info] [pid 9162:tid 140326166714128] [remote 127.0.0.1:8532] 
AH02276: Certificate Verification: Error (3): unable to get certificate 
CRL [subject: 
emailAddress=test-...@httpd.apache.org,CN=localhost,OU=httpd-test/rsa-test,O=ASF,L=San 
Francisco,ST=California,C=US / issuer: 
emailAddress=test-...@httpd.apache.org,CN=ca,OU=httpd-test,O=ASF,L=San 
Francisco,ST=California,C=US / serial: 0C / notbefore: Jul 30 23:29:05 
2020 GMT / notafter: Jul 30 23:29:05 2021 GMT]


[ssl:info] [pid 9162:tid 140326149928720] [client 

Re: Pending fixes or reroll? Was: [RESULT] [VOTE] Release httpd-2.4.45

2020-07-31 Thread Rainer Jung
Since here were lots of "good-to-go, let's roll" one note though: the 
current CHANGES does not yet a section for 2.4.46, cause normally 
APLOGNO doesn't get noted on CHANGES. In this case it might be something 
like


  *) mod_proxy_fcgi: Fix build on Windows.

and credits for Eric (who fixed it on trunk) oand/or Christophe, who 
backported it.


Best regards,

Rainer

Am 31.07.2020 um 15:36 schrieb Rainer Jung:
Since there wasn't yet any reaction to Daniel's question: Is anybody 
right now working on more warnings fixes for Windows?


The most prominent one (missing APLOGNo number = missing macro argument) 
IMHO was already fixed by Christophe in r1880438. Anything else worth 
waiting for or are we (is Daniel) good to go for 2.4.46?


Concerning lua I'd say the fix(es) for 5.4.0 need a bit more testing.

Regards,

Rainer

Am 30.07.2020 um 22:26 schrieb Daniel Ruggeri:

Hi, all;
    I thank everyone for their testing and quick feedback. While we had
enough votes and positive feedback, we have some easily fixable warnings
which have precedence for holding up a release.

    To that end, I will close this vote and declare 2.4.45 dead on the 
vine.


    I will roll 2.4.46 when we are all buttoned up with the warnings.


Pending fixes or reroll? Was: [RESULT] [VOTE] Release httpd-2.4.45

2020-07-31 Thread Rainer Jung
Since there wasn't yet any reaction to Daniel's question: Is anybody 
right now working on more warnings fixes for Windows?


The most prominent one (missing APLOGNo number = missing macro argument) 
IMHO was already fixed by Christophe in r1880438. Anything else worth 
waiting for or are we (is Daniel) good to go for 2.4.46?


Concerning lua I'd say the fix(es) for 5.4.0 need a bit more testing.

Regards,

Rainer

Am 30.07.2020 um 22:26 schrieb Daniel Ruggeri:

Hi, all;
    I thank everyone for their testing and quick feedback. While we had
enough votes and positive feedback, we have some easily fixable warnings
which have precedence for holding up a release.

    To that end, I will close this vote and declare 2.4.45 dead on the vine.

    I will roll 2.4.46 when we are all buttoned up with the warnings.


Re: Providing near-time averaged monitoring data for mod_systemd and mod_status

2020-04-27 Thread Rainer Jung

Am 27.04.2020 um 15:57 schrieb Joe Orton:

On Sat, Apr 25, 2020 at 08:10:40PM +0200, Rainer Jung wrote:

Patch available at

home.apache.org/~rjung/patches/httpd-trunk-mon-snaps-v1_2.patch


Very nice!  +1 from me.

Does the times_per_thread logic still make any sense?  It's always been
wrong for Linux AFAICT so maybe can just be dropped.


I will do some code archeology to find out who added it and why.


A minor nit, I think the snap_index should be read/written using
apr_atomic_t since it's going to be accessed concurrently across
threads?


Yup. Since we only have concurrent reads, it should suffice to check the 
value with apr_atomic_read32 and then use apr_atomic_set32 to set it the 
0 or 1.


I wonder how we best make sure, that the new monitor hook is not called 
from modules? Currently it is no AP_DECLARE'd but contained in httpd.h. 
And the hook registration is in server/core.c, but the implementation in 
server/scoreboard.c. No idea (not tried) whether it would be better to 
have it in server/core.c as a static function or whether that's not 
needed and it would suffice to move the function declaration to a 
private header file.


Thanks and regards,

Rainer


Some notes:

- in order to use the data from mod_systemd (monitor hook), which runs in
the maim process, and also from mod_status, so from child processes, it
needs to sit in shared memory. Since most of the data is tightly coupled to
the scoreboard, I added the new cached data to the global part of the
scoreboard.

- it was then IMHO a good fit to move the existing ap_get_sload(ap_sload_t
*ld) from server/util.c to server/scoreboard.c.

- ap_sload_t is used as a collection of data which can be used to generate
averaged data from a pair of ap_sload_t structs. It was extended to also
contain the total request duration plus total usr and sys CPU and the
timestamp when the data was taken. So it should now contain all data from
which mod-systemd and mod_status currently like to display derived averaged
values.

- busy and idle in ap_sload_t have been changed from int to float, because
they were already only used a percentages. IMHO it doesn't make sense to
express idle and busy as percentages based on a total of busy plus idle,
because that sum is not a meaningful total. The server can grow by creating
new processes and suddenly your busy percentage might shrink, although the
absolute number of busy threads increases. That's counter intuitive. So I
added the unused process slots to the total count, and we have three
counters that sum up to this total, busy, idle and dead. We need a better
name than "dead" for these unused process slots that might get used later.
"unused" is to close to "idle" and I don't have a good name yet.

- the new ap_mon_snap_t contains a pointer to an ap_sload_t, six averaged
values generated from two ap_sload_t and a member that conatins the time
delta between those two ap_sload_t. The ap_sload_t referenced by
ap_mon_snap_t contains the data at time t1. During the next monitor run, new
t1 data will be collected and the previous t1 data will be used as t0 data
to generate the new averages.

- the scoreboard contains two ap_mon_snap_t (plus the two ap_sload_t
referenced by them) to be able to update one of them without breaking access
by consumers to the other one. After the update the roles get switched.

- both structs, ap_sload_t and ap_mon_snap_t are declared in httpd.h,
because ap_sload_t already had been there. t might be a better fit to move
them to scoreboard.h, but I'm not sure about that and I don't know if that
would be allowed by the compatibility rules.

- also in httpd.h there are now three new function declarations. Two only
used by server/core.c as hook functions:

   int ap_scoreboard_monitor(apr_pool_t *p, server_rec *s);
   void ap_scoreboard_child_init(apr_pool_t *p, server_rec *s);

and one for public consumption:

   AP_DECLARE(void) ap_get_mon_snap(ap_mon_snap_t *ms);

- mod_systemd and mod_status now call ap_get_mon_snap() to retrieve the
latest averaged data. mod_status still uses the scoreboard in addition to
retrieve up-to-date current scalar metrics. Small adjustments to the
mod_status output plus additions to mod_systemd notification messages. Tne
STATUS in the notification of mod_systemd now looks liek this:

STATUS=Pct Idle workers 28.4; Pct Busy workers 10.6; Pct Dead workers 60.9;
Requests/sec: 5030; Bytes served/sec: 2.9MB/sec; Bytes served/request: 596
B/req; Avg Duration(ms): 5.78; Avg Concurrency: 29.1; Cpu Pct: 40.5

- scoreboard.c changes:

   - take over ap_get_sload(ap_sload_t *sl) from server/util.c.
 Added copied code from mod_status to that function to also add up the
request duration and the usr and sys CPU times.

   - ap_scoreboard_monitor() runs in the monitor hook. It calls
ap_get_sload() and then a static utility function calc_mon_data() to
calculate the new averages.

- Minor mmn change not yet part of the patch.

It compiles fine (mai

Re: Providing near-time averaged monitoring data for mod_systemd and mod_status

2020-04-27 Thread Rainer Jung

Am 27.04.2020 um 17:28 schrieb Yann Ylavic:

On Mon, Apr 27, 2020 at 5:18 PM Rainer Jung  wrote:


Hi Yann,

Am 27.04.2020 um 16:40 schrieb Yann Ylavic:

Hi Rainer,

On Mon, Apr 27, 2020 at 3:17 PM Rainer Jung  wrote:


Thanks for this.
Could you please create this as a PR on github as well? This ensures that all 
the Travis tests are run for your patch.


Thanks Rüdiger. Done and indeed Travis found one not that I fixed but we
need to discuss.


Thanks Rainer, looks very nice.

Could we avoid the "ap_sload_t *sload" pointer in ap_mon_snap_t somehow?
I'm asking because snap0 and snap1 in global_score (SHM) end up
containing a pointer now.


My initial version had the fields of ap_sload_t directly inside
ap_mon_snap_t instead of the pointer.

I changed it, because

- it allowed to more safely copy the whole ap_sload_t without relying on
keeping two struct definitions in sync

- it allowed to extend ap_sload_t and ap_mon_snap_t at their respective
ends and stay compatible.

You have a problem with the pointers because of possible alignment
issues or what kind of trouble might we run into? Or is it just hard to
understand (and more comments in the code will help)?


My concern is more about security, e.g. CVE-2019-0211 kind of things,
with an unprivileged user (child) playing with this pointer and making
the main process (monitor?) at best crash, at worst do anything they
want (root privilege escalation).


I see. Although I don't see a way how we would execute code at the 
derefenced address, as a safety guard for future code changes we can 
easily avoid using the pointers in the scoreboard and instead use sload0 
/ snap0 and sload1 / snap1 when doing the calculations. Since the 
pointers (after init) would have been constant, we can hard code it into 
trivial logic instead.


One could then use a separate type for snap0 and snap1 in the 
scoreboard, but I would find it better to stay with the ap_mon_snap_t 
type, even if we do not use the pointer there.


For the struct passed in from the consumer, I would still use the 
pointer as-is.


Will provide a change on top of the PR for further discussion.

Thanks for the heads up. Didn't think about using the scoreboard and the 
monitor hook running in the parent havig security implications. One 
really needs to be careful about assumptions for the scoreboard data!


Regards,

Rainer


Re: Providing near-time averaged monitoring data for mod_systemd and mod_status

2020-04-27 Thread Rainer Jung

Hi Yann,

Am 27.04.2020 um 16:40 schrieb Yann Ylavic:

Hi Rainer,

On Mon, Apr 27, 2020 at 3:17 PM Rainer Jung  wrote:


Thanks for this.
Could you please create this as a PR on github as well? This ensures that all 
the Travis tests are run for your patch.


Thanks Rüdiger. Done and indeed Travis found one not that I fixed but we
need to discuss.


Thanks Rainer, looks very nice.

Could we avoid the "ap_sload_t *sload" pointer in ap_mon_snap_t somehow?
I'm asking because snap0 and snap1 in global_score (SHM) end up
containing a pointer now.


My initial version had the fields of ap_sload_t directly inside 
ap_mon_snap_t instead of the pointer.


I changed it, because

- it allowed to more safely copy the whole ap_sload_t without relying on 
keeping two struct definitions in sync


- it allowed to extend ap_sload_t and ap_mon_snap_t at their respective 
ends and stay compatible.


You have a problem with the pointers because of possible alignment 
issues or what kind of trouble might we run into? Or is it just hard to 
understand (and more comments in the code will help)?


Or because it is harder for consumers to use the API in a correct way 
(the pointer in ap_mon_snap_t might be uninitialized)? In the latter 
case: one could also see this as a feature instead of as a bug: as long 
as one is only interested in the averaged data and not in the sload 
data, you can pass in an ap_mon_snap_t with a null pointer to the 
ap_sload_t.


Regards,

Rainer


Re: Providing near-time averaged monitoring data for mod_systemd and mod_status

2020-04-27 Thread Rainer Jung

Am 27.04.2020 um 08:57 schrieb Ruediger Pluem:



On 4/25/20 8:10 PM, Rainer Jung wrote:

Patch available at

home.apache.org/~rjung/patches/httpd-trunk-mon-snaps-v1_2.patch


Thanks for this.
Could you please create this as a PR on github as well? This ensures that all 
the Travis tests are run for your patch.


Thanks Rüdiger. Done and indeed Travis found one not that I fixed but we 
need to discuss.


I changed the data type of the idle and busy fields in the sload struct 
from int to float, because they were used as floats. But that broke 
mod_headers which uses the data and will break other consumers as well.


For compatibility reasons, we could keep the int type though.

Regards,

Rainer


Providing near-time averaged monitoring data for mod_systemd and mod_status

2020-04-25 Thread Rainer Jung

Patch available at

home.apache.org/~rjung/patches/httpd-trunk-mon-snaps-v1_2.patch

Some notes:

- in order to use the data from mod_systemd (monitor hook), which runs 
in the maim process, and also from mod_status, so from child processes, 
it needs to sit in shared memory. Since most of the data is tightly 
coupled to the scoreboard, I added the new cached data to the global 
part of the scoreboard.


- it was then IMHO a good fit to move the existing 
ap_get_sload(ap_sload_t *ld) from server/util.c to server/scoreboard.c.


- ap_sload_t is used as a collection of data which can be used to 
generate averaged data from a pair of ap_sload_t structs. It was 
extended to also contain the total request duration plus total usr and 
sys CPU and the timestamp when the data was taken. So it should now 
contain all data from which mod-systemd and mod_status currently like to 
display derived averaged values.


- busy and idle in ap_sload_t have been changed from int to float, 
because they were already only used a percentages. IMHO it doesn't make 
sense to express idle and busy as percentages based on a total of busy 
plus idle, because that sum is not a meaningful total. The server can 
grow by creating new processes and suddenly your busy percentage might 
shrink, although the absolute number of busy threads increases. That's 
counter intuitive. So I added the unused process slots to the total 
count, and we have three counters that sum up to this total, busy, idle 
and dead. We need a better name than "dead" for these unused process 
slots that might get used later. "unused" is to close to "idle" and I 
don't have a good name yet.


- the new ap_mon_snap_t contains a pointer to an ap_sload_t, six 
averaged values generated from two ap_sload_t and a member that conatins 
the time delta between those two ap_sload_t. The ap_sload_t referenced 
by ap_mon_snap_t contains the data at time t1. During the next monitor 
run, new t1 data will be collected and the previous t1 data will be used 
as t0 data to generate the new averages.


- the scoreboard contains two ap_mon_snap_t (plus the two ap_sload_t 
referenced by them) to be able to update one of them without breaking 
access by consumers to the other one. After the update the roles get 
switched.


- both structs, ap_sload_t and ap_mon_snap_t are declared in httpd.h, 
because ap_sload_t already had been there. t might be a better fit to 
move them to scoreboard.h, but I'm not sure about that and I don't know 
if that would be allowed by the compatibility rules.


- also in httpd.h there are now three new function declarations. Two 
only used by server/core.c as hook functions:


  int ap_scoreboard_monitor(apr_pool_t *p, server_rec *s);
  void ap_scoreboard_child_init(apr_pool_t *p, server_rec *s);

and one for public consumption:

  AP_DECLARE(void) ap_get_mon_snap(ap_mon_snap_t *ms);

- mod_systemd and mod_status now call ap_get_mon_snap() to retrieve 
the latest averaged data. mod_status still uses the scoreboard in 
addition to retrieve up-to-date current scalar metrics. Small 
adjustments to the mod_status output plus additions to mod_systemd 
notification messages. Tne STATUS in the notification of mod_systemd now 
looks liek this:


STATUS=Pct Idle workers 28.4; Pct Busy workers 10.6; Pct Dead workers 
60.9; Requests/sec: 5030; Bytes served/sec: 2.9MB/sec; Bytes 
served/request: 596 B/req; Avg Duration(ms): 5.78; Avg Concurrency: 
29.1; Cpu Pct: 40.5


- scoreboard.c changes:

  - take over ap_get_sload(ap_sload_t *sl) from server/util.c.
Added copied code from mod_status to that function to also add up 
the request duration and the usr and sys CPU times.


  - ap_scoreboard_monitor() runs in the monitor hook. It calls 
ap_get_sload() and then a static utility function calc_mon_data() to 
calculate the new averages.


- Minor mmn change not yet part of the patch.

It compiles fine (maintainer mode) on RHEL 7 x86_64 and on Solaris 10 
Sparc and I did some tests with mod_status and mod_systemd.


Regards,

Rainer

Am 24.04.2020 um 18:32 schrieb Rainer Jung:

Am 24.04.2020 um 16:21 schrieb Joe Orton:

On Fri, Apr 24, 2020 at 12:17:19PM +0200, Rainer Jung wrote:

Thinking further: I think it would make sense to have a module or core
implement the monitor hook to generate that derived data (requests/sec,
bytes/sec, durationMs/request, avgConcurrency) in the last monitor 
interval
and to provide that data to consumers like mod_systemd or - new - 
mod_status
- instead of the long term averages since start. It could probably be 
added
to the code that already provides "sload". That way mod_status would 
also

profit from the more precise average values (taken over the last monitor
interval).


I definitely like the patch, it has bothered me that the "per second"
stats are not very useful but wasn't sure how to make it better.

This is also an interesting idea.

So you would suggest having a new monit

Re: mod_systemd suggestion

2020-04-24 Thread Rainer Jung

Am 24.04.2020 um 16:21 schrieb Joe Orton:

On Fri, Apr 24, 2020 at 12:17:19PM +0200, Rainer Jung wrote:

Thinking further: I think it would make sense to have a module or core
implement the monitor hook to generate that derived data (requests/sec,
bytes/sec, durationMs/request, avgConcurrency) in the last monitor interval
and to provide that data to consumers like mod_systemd or - new - mod_status
- instead of the long term averages since start. It could probably be added
to the code that already provides "sload". That way mod_status would also
profit from the more precise average values (taken over the last monitor
interval).


I definitely like the patch, it has bothered me that the "per second"
stats are not very useful but wasn't sure how to make it better.

This is also an interesting idea.

So you would suggest having a new monitor hook which runs REALLY_LAST in
the order, calls ap_get_sload() and stores it in a global, and then we'd
have an ap_get_cached_sload() (or whatever) which gives you the cached
data from the last iteration?  Or are you thinking of a more
sophisticated API which does the "diff" between intervals internally?


Thanks for the positive feedback.

The averaged metrics IMHO only make sense as cached data, updated in 
regular intervals and provided for use by various modules (probably only 
mod_systemd and mod_status).


I would like to provide the already averaged data in a struct, each 
metric as a float or double. The bytes/request probably not already 
human readably scaled, because it makes its use less flexible. Since we 
already also have the absolute counters at that point, we can easily add 
them to the same struct as 32 or 64 bit counters and return a consistent 
set of data (five old values, five new values, five averages and two 
time stamps). [idle(32), busy(32), requests(64), bytes(64), 
duration(64); req/s, bytes/s, bytes/req, dur/s, dur/req]. So consumers 
needing a consistent view can get it.


Even more so since the absolute metrics are currently not cheap to 
access. We collect all of them by iterating over the scoreboard and 
summing up. By adding them to the cached data, the consuming code could 
decide, whether such near-time data is good enough or it needs to 
acquire new and curent counters. For mod_systemd, cached data (10 second 
interval) might be OK.


For some modules - like mod_status - cached averages are fine, but I 
think the counters should be correct for the point in time the status 
request was handled by the module. So the scoreboard statistics code in 
mod_status unfortunately would not go away.´, but the data quality for 
the averages would become better.


Implementation wise I am thinking about adding

  ap_hook_monitor(mon_avg_monitor, NULL, NULL, ???);

to server/util.c, which calculates the new averages and

  ap_get_mon_avg(ap_mon_avg_t *ma)

which returns the four averages in a struct similar to the existing 
ap_get_loadavg() and ap_get_sload().


We might have a little hassle to make the statistics update 
atomic/thread-safe (eg. two instances of the internal data struct, so 
that we only need to make the switch between them after the new 
calculation atomic).


About REALLY_LAST: why last? If other modules collect data via this API 
and wasn't to do it in the monitor hook as well, shouldn't run the 
caching of data REALLY_FIRST, so you get the new averages?


I'll try to draft and test something along these lines later today. Fun 
stuff. And more comments are very welcome.


And I like, that mod_status will profit by showing betrer averages as well.

Regards,

Rainer


Am 23.04.2020 um 21:29 schrieb Rainer Jung:

Hi all,

triggered by the new mod_systemd I drafted a patch to enhance the
monitoring data it provides during the monitor hook run.

Currently it publishes important data, like idle and busy slots and
total request count, but also not so useful info like requests/second
and bytes/second as a long term average (since start). These two figues
tend to become near constant after a longer time of operation.

Since the monitor hook of the module always seems to run in the same
(parent) process, it is easy to remember the previous request and byte
count data and average only over the last monitor hook interval. This
should give more meaningful data. And is a change local to mod_systemd.

In addition we have a third metric available in the scoreboard, namely
the total request duration. From that we can get the average request
duration and the average request concurrency. This part also needs a
change to the sload structure. Maybe we need a minor MMN bump for that.

I scetched a patch under

home.apache.org/~rjung/patches/httpd-trunk-mod_systemd-interval-stats.patch

Any comments, likes or dislikes?

Thanks and regards,

Rainer


Re: mod_systemd suggestion

2020-04-24 Thread Rainer Jung
Thinking further: I think it would make sense to have a module or core 
implement the monitor hook to generate that derived data (requests/sec, 
bytes/sec, durationMs/request, avgConcurrency) in the last monitor 
interval and to provide that data to consumers like mod_systemd or - new 
- mod_status - instead of the long term averages since start. It could 
probably be added to the code that already provides "sload". That way 
mod_status would also profit from the more precise average values (taken 
over the last monitor interval).


Regards,

Rainer

Am 23.04.2020 um 21:29 schrieb Rainer Jung:

Hi all,

triggered by the new mod_systemd I drafted a patch to enhance the 
monitoring data it provides during the monitor hook run.


Currently it publishes important data, like idle and busy slots and 
total request count, but also not so useful info like requests/second 
and bytes/second as a long term average (since start). These two figues 
tend to become near constant after a longer time of operation.


Since the monitor hook of the module always seems to run in the same 
(parent) process, it is easy to remember the previous request and byte 
count data and average only over the last monitor hook interval. This 
should give more meaningful data. And is a change local to mod_systemd.


In addition we have a third metric available in the scoreboard, namely 
the total request duration. From that we can get the average request 
duration and the average request concurrency. This part also needs a 
change to the sload structure. Maybe we need a minor MMN bump for that.


I scetched a patch under

home.apache.org/~rjung/patches/httpd-trunk-mod_systemd-interval-stats.patch

Any comments, likes or dislikes?

Thanks and regards,

Rainer


mod_systemd suggestion

2020-04-23 Thread Rainer Jung

Hi all,

triggered by the new mod_systemd I drafted a patch to enhance the 
monitoring data it provides during the monitor hook run.


Currently it publishes important data, like idle and busy slots and 
total request count, but also not so useful info like requests/second 
and bytes/second as a long term average (since start). These two figues 
tend to become near constant after a longer time of operation.


Since the monitor hook of the module always seems to run in the same 
(parent) process, it is easy to remember the previous request and byte 
count data and average only over the last monitor hook interval. This 
should give more meaningful data. And is a change local to mod_systemd.


In addition we have a third metric available in the scoreboard, namely 
the total request duration. From that we can get the average request 
duration and the average request concurrency. This part also needs a 
change to the sload structure. Maybe we need a minor MMN bump for that.


I scetched a patch under

home.apache.org/~rjung/patches/httpd-trunk-mod_systemd-interval-stats.patch

Any comments, likes or dislikes?

Thanks and regards,

Rainer



Re: mod_http2 crashes libhttpd.dll in 2.4.43

2020-04-16 Thread Rainer Jung
If I get this right, there is an element in elts, that has a valid 
string key ("H2_STREAM_ID") bis a NULL value (2nd screenshot) and the 
condition check in line 527 of the first screen shot checks key against 
NULL and empty, but value only against empty but not against NULL. So 
the empty check derefences NULL.


Nut sure what the correct fix is:

- make sure the H2_STREAM_IS value is never NULL (but maybe empty)
- add NULL check for the value to the list of checks in 527
- something else

At least the debug info provided by Steffen seems to be a good fit to 
the stream id handling changes in the revision in question (but i have 
not checked, whether the NULL is new). I think a fix is close :)


Thanks and regards,

Rainer

Am 16.04.2020 um 10:12 schrieb Steffen:

More info.

See CallStack

  www.apachelounge.com/download/VS16/modules/CallStack.png

and

  www.apachelounge.com/download/VS16/modules/autos.png

Below we had:

libhttpd!ap_get_server_built+0x5d9
mod_cgi+0x14aa
libhttpd!ap_run_handler+0x35
libhttpd!ap_invoke_handler+0x10f
libhttpd!ap_internal_redirect_handler+0x29a
libhttpd!ap_process_request+0xf
mod_http2+0x188ef
libhttpd!ap_run_process_connection+0x35
mod_http2+0x185ba
mod_http2+0x1c36e
ucrtbase!beginthreadex+0x142
kernel32!BaseThreadInitThunk+0x14
ntdll!RtlUserThreadStart+0x21


Steffen
On Tuesday 14/04/2020 at 14:13, Eric Covener wrote:

On Tue, Apr 14, 2020 at 8:09 AM Ruediger Pluem  wrote:




On 4/14/20 12:22 PM, Steffen wrote:



This is the post above of backtrace


Thanks.



By accident I've seen that Perl comes with GDB. This might help as well.
I called httpd.exe from GDB with "-X -e debug" and then called a 
Perl URL in the browser.


Excerpt below:



Somehow the below wasn't visible in the original mail.


Thread 100 received signal SIGSEGV, Segmentation fault.
[Switching to Thread 4936.0x23e0]
0x7ffbe57515d9 in libhttpd!ap_get_server_built () from 
X:\Apps\Apache24\bin\libhttpd.dll

(gdb) bt
#0 0x7ffbe57515d9 in libhttpd!ap_get_server_built () from 
X:\Apps\Apache24\bin\libhttpd.dll

#1 0x7ffbe44d14aa in ?? () from X:\Apps\Apache24\modules\mod_cgi.so
#2 0x7ffbe575ee85 in libhttpd!ap_run_handler () from 
X:\Apps\Apache24\bin\libhttpd.dll
#3 0x7ffbe575da7f in libhttpd!ap_invoke_handler () from 
X:\Apps\Apache24\bin\libhttpd.dll
#4 0x7ffbe575a62a in libhttpd!ap_internal_redirect_handler () 
from X:\Apps\Apache24\bin\libhttpd.dll
#5 0x7ffbe575a6af in libhttpd!ap_process_request () from 
X:\Apps\Apache24\bin\libhttpd.dll
#6 0x7ffbe22888ef in ?? () from 
X:\Apps\Apache24\modules\mod_http2.so
#7 0x7ffbe5761545 in libhttpd!ap_run_process_connection () from 
X:\Apps\Apache24\bin\libhttpd.dll
#8 0x7ffbe22885ba in ?? () from 
X:\Apps\Apache24\modules\mod_http2.so
#9 0x7ffbe228c36e in ?? () from 
X:\Apps\Apache24\modules\mod_http2.so
#10 0x7ffbe9e30e72 in ucrtbase!_beginthreadex () from 
C:\Windows\System32\ucrtbase.dll
#11 0x7ffbea107bd4 in KERNEL32!BaseThreadInitThunk () from 
C:\Windows\System32\kernel32.dll
#12 0x7ffbebecced1 in ntdll!RtlUserThreadStart () from 
C:\Windows\SYSTEM32\ntdll.dll

#13 0x in ?? ()
Backtrace stopped: previous frame inner to this frame (corrupt stack?)
(gdb)




Unfortunately this stacktrace does not help. One reason might be that 
the debugging symbols are missing.
It is very strange that it segfaults in ap_get_server_built, a simple 
function just returning a pointer
to a static string constant. Furthermore ap_get_server_built is not 
called by mod_cgi.
Can the crash be repeated against a binary with debugging symbols 
that are then used to generate the stacktrace?
As I am not a Windows guy, I unfortunately cannot provide any 
instructions how to do this.


My experience on windows is that if the PDB's are not 110% right you
will get all kinds of misleading stuff above the first ?? in the
displayed backtrace.


Re: mod_http2 crashes libhttpd.dll in 2.4.43

2020-04-14 Thread Rainer Jung

Hi Steffen,

I didn't find a stack either, neither in the mail thread here nor in the 
one on you forum. Just the error log lines and the clear text for the 
windows error code.


Regards,

Rainer

Am 14.04.2020 um 11:51 schrieb Steffen:

Few posts above there is a GDB back trace.

On Tuesday 14/04/2020 at 11:38, Ruediger Pluem wrote:


Did I miss it, or isn't there any stacktrace posted from the Windows 
crashes yet?


Regards

Rüdiger


Re: mod_http2 crashes libhttpd.dll in 2.4.43

2020-04-14 Thread Rainer Jung

Sorry, that was a bit to fast: looking at the revision, he references file

modules/http2/h2_bucket_beam.h

But in that revision the change was the removal of a struct member at 
the end of the struct. I don't see, how that could lead to a crash in 
case an old header file was used, that would sill contain that - now 
unused - struct member?


Regards,

Rainer

Am 14.04.2020 um 11:05 schrieb Rainer Jung:

Hi Stefen,

I think Stefan refers to his "somehow was partially using the old header 
file". So during the build, there should be no other (from other 
versions) header files from APR, APR-UTIL oder HTTPD anywhere on the 
build system where the build process might find and use them.


He was assuming, that the build that had the crashes was possibly done 
on a system, which did not fulfil this requirement.


Regards,

Rainer

Am 14.04.2020 um 10:13 schrieb Steffen:


What do you mean with a clean rebuild ?

r1874909 (1.15.8) is the one from  httpd 2.4.43 GA.


On Tuesday 14/04/2020 at 09:38, Stefan Eissing wrote:
In that revision, I see a change in the header file structure. If 
your build of r1874909 is not clean but somehow was partially using 
the old header file, things might get misaligned.


The other changes in that revision look rather unsuspicious to me.

Steffen: could you make a clean rebuild of mod_http2 for mdrmdr to 
verify? Thanks!


Cheers, Stefan


Am 12.04.2020 um 15:57 schrieb mdrmdr :

With the help of Steffen (he built the Windows binaries for me) I 
can report:


r1861247 - works
r1864126 - works
r1872230 - works
r1873368 - works
r1874286 - works
r1874347 - works
r1874909 (=2.4.43) - fails!



--
Sent from: 
http://apache-http-server.18135.x6.nabble.com/Apache-HTTP-Server-Dev-f4771363.html 


Re: mod_http2 crashes libhttpd.dll in 2.4.43

2020-04-14 Thread Rainer Jung

Hi Stefen,

I think Stefan refers to his "somehow was partially using the old header 
file". So during the build, there should be no other (from other 
versions) header files from APR, APR-UTIL oder HTTPD anywhere on the 
build system where the build process might find and use them.


He was assuming, that the build that had the crashes was possibly done 
on a system, which did not fulfil this requirement.


Regards,

Rainer

Am 14.04.2020 um 10:13 schrieb Steffen:


What do you mean with a clean rebuild ?

r1874909 (1.15.8) is the one from  httpd 2.4.43 GA.


On Tuesday 14/04/2020 at 09:38, Stefan Eissing wrote:
In that revision, I see a change in the header file structure. If your 
build of r1874909 is not clean but somehow was partially using the old 
header file, things might get misaligned.


The other changes in that revision look rather unsuspicious to me.

Steffen: could you make a clean rebuild of mod_http2 for mdrmdr to 
verify? Thanks!


Cheers, Stefan


Am 12.04.2020 um 15:57 schrieb mdrmdr :

With the help of Steffen (he built the Windows binaries for me) I can 
report:


r1861247 - works
r1864126 - works
r1872230 - works
r1873368 - works
r1874286 - works
r1874347 - works
r1874909 (=2.4.43) - fails!



--
Sent from: 
http://apache-http-server.18135.x6.nabble.com/Apache-HTTP-Server-Dev-f4771363.html


Re: Building from svn on MacOS

2020-04-13 Thread Rainer Jung

Am 13.04.2020 um 23:27 schrieb William A Rowe Jr:
On Mon, Apr 13, 2020 at 4:21 PM Christopher Schultz 
mailto:ch...@christopherschultz.net>> wrote:


-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

William,

 >> I'm having some trouble building 2.4.x directly from svn.
 >>
 >> MacOS 10.14.6 (Mojave)
 >
 > I note you mentioned apr 1.7.0. If you grab and pre build apr, and
 > then apr-util (and openssl and anything else you want to refresh)
 > or install the compiled system package, it should work. Point at
 > them --with-apr plus --with-aprutil.

I'm using brew which is like the missing package manager for macos.
I've installed apr and apr-util which I thikn are both binary
packages. I reconfigured with:

$ ./buildconf --with-apr=/usr/local/Cellar/apr/1.7.0/bin/apr-1-config
- --with-apr-util=/usr/local/Cellar/apr-util/1.6.1_3/bin/apu-1-config

I get this output:

using apr-config version 1.7.0
./buildconf: line 249: cd:
/usr/local/Cellar/apr-util/1.6.1_3/bin/apu-1-config: Not a directory
copying build files


That's your answer. It wants the parent path of the apr/apr-util 
installations,
not the name of the apr-1-config file. It will work out 
bin/apr-1-config... etc.


The script looks like it should work using the path to apr-1-config, but 
if used in that way it unfortunately does not do what it announces, 
namely ignoring the apr-util setting. What is definitely not suppported 
is using a path to apu-1-config. This only works for apr, not apr-util.


I recommend not trying to run buildconf against installed (binary) apr / 
apr-util but instead against an unpacked source download of these two. 
Run it with giving the path to the unpacked sources of the two. The 
script tries to copy a few files from the source tree and it is unclear, 
whether those files actually get packaged by people providing a binary 
distribution.


Regards,

Rainer



Re: Solaris, prefork, accept mutex and mod_ext_filter (Was: Prefork MPM and mod_watchdog)

2020-03-30 Thread Rainer Jung

Am 30.03.2020 um 11:28 schrieb Joe Orton:

On Mon, Mar 30, 2020 at 12:31:05AM +0200, Rainer Jung wrote:

I can now see the same problem on Linux (eg. RHEL 7, SLES 12 and SLES 15)
when doing testing for 2.4.43. I think it is not a regression and for me it
is not a showstopper, but something that would be nice to fix. Symptom is

[Sat Mar 28 15:53:21.121461 2020] [mpm_prefork:debug] [pid 4574]
prefork.c(919): AH00165: Accept mutex: pthread (default: pthread)
[Sat Mar 28 15:58:19.251858 2020] [mpm_prefork:emerg] [pid 6509] (22)Invalid
argument: AH00144: couldn't grab the accept mutex
[Sat Mar 28 15:58:20.624995 2020] [mpm_prefork:emerg] [pid 4902] (22)Invalid
argument: AH00146: couldn't release the accept mutex
[Sat Mar 28 15:58:21.410552 2020] [:emerg] [pid 4574] AH02818: MPM run
failed, exiting

happening during t/modules/ext_filter.t and as a result all httpd processes
are gone.

Although it happens only rarely, I think it does not happen when using APR
1.6, but only for 1.7 (1.7.0 and also for latest head). The accept mutex is
pthread based. There are changes in APR 1.7 for those.


This is interesting, I saw some similar mutex state snafu (Fedora 31)
but assumed it was some kind of PEBKAC issue and moved on.  You have
only seen this with prefork?


Yes, only with prefork. If the root cause would also apply for 
multi-threaded MPMs, then the escalation might be different, ie. not 
leading to all child processes exiting. I can not rule out this.



May be completely unrelated but since I added a grep for child segfaults
in error_log that has triggered with prefork at least once:

https://travis-ci.org/github/apache/httpd/jobs/665874906


That looks more like my rarely observed prefork shutdown crashes when 
mod_watchdog shuts down after some other deinitialization that happened 
before.


Regards,

Rainer


Regards, Joe


Since I only observe it for mod_ext_filter, there should be some dependency
with the forked perl process.

I didn't yet have the opportunity to check, how much of the follwing
description for Solaris also holds for Linux.

Regards,

Rainer

Am 03.02.2019 um 13:30 schrieb Rainer Jung:

I can now frequently reproduce running t/modules/ext_filter.t only. I
stripped the reproducer down to the part of t/modules/ext_filter.t where
it runs

POST "/apache/extfilter/out-limit/modules/cgi/perl_echo.pl", content =>
"foo and bar\n"

10 times in a row. If I increase the iteration count slightly to 20, the
problem happens nearly every time. I also replaced perl_echo.pl and
eval-cmd.pl by small C programs doing the echo and the s/foo/bar/ and
can still reproduce.

This test involved mod_ext_filter and LimitRequestBody.

It seems I can not reproduce, if I shorten the POST body, so that it no
longer hits the LimitRequestBody 6 configured for
/apache/extfilter/out-limit/.

What happens, but I do not understand is:

- the first few requests are handled by two children in an alternating
way. I can see the accept mutex being passed between these two children
using lwp_mutex_timedlock(ADDRESS, 0x) and
lwp_mutex_unlock(ADDRESS) using truss (Solaris variant of strace). So
Solaris seems to implement the pthread mutexes via these lwp mutexes.

- after a few requests alternating between the two children, something
strange happens:

    - child A returns from port_getn() for 22 (probably part of the
accept() impl)
    - child A calls accept()
    - child A unlocks the accept mutex using lwp_mutex_unlock()
    - child B locks the accept mutex using lwp_mutex_timedlock()
    - child B calls port_associate for 22 (probably part of accept() impl)
    - child A handles the request
! - child A also calls port_associate for 22 (no sign of lock acquire)
! - child A returns from port_getn() for 22
    - child A calls accept() and starts to handle the connection
! - child B also returns from port_getn() for 22
! - child B gets EAGAIN for the accept()
    - child B calls port_associate for 22
    - child A handles the request
! - child A gets EDEADLK for pthread_mutex_lock() (this is from the
error_log; there's no system call for this instance of
pthread_mutex_lock() in the truss output). EDEADLK for
pthread_mutex_lock() means that this process already has that lock.
    - child B returns from port_getn() for 22
    - child B calls accept() and starts to handle the connection
    - child A exits (now B is the only child left)
    - child B proceeds request handling
    - child B does all further port_associate(), port_getn(), and
accept() plus connection and request handling. No more
lwp_mutex_timedlock() or lwp_mutex_unlock() for B, maybe due to some
optimization when only one process for a lock is left.


It is quite possible, that there is an underlying lwp_mutex or portfs
bug here. But it is strange, that this only comes up when used with
mod_ext_filter in combination with LimitRequestBody.

There was the fix

https://bz.apache.org/bugzilla/show_bug.cgi?id=60375

but I don't see, h

Re: [VOTE] Release httpd-2.4.43

2020-03-29 Thread Rainer Jung

Am 26.03.2020 um 15:50 schrieb Daniel Ruggeri:

Hi, all;
    Please find below the proposed release tarball and signatures:
https://dist.apache.org/repos/dist/dev/httpd/

I would like to call a VOTE over the next few days to release this
candidate tarball as 2.4.43:
[X] +1: It's not just good, it's good enough!
[ ] +0: Let's have a talk.
[ ] -1: There's trouble in paradise. Here's what's wrong.

The computed digests of the tarball up for vote are:
sha1: 15d8605b094dfe5e283cd9e90770368dd14e26f2 *httpd-2.4.43.tar.gz
sha256: 2624e92d89b20483caeffe514a7c7ba93ab13b650295ae330f01c35d5b50d87f
*httpd-2.4.43.tar.gz
sha512:
d9879b8f8ef7d94dee1024e9c25b56d963a3b072520878a88a044629ad577c109a5456791b39016bf4f6672c04bf4a0e5cfd32381211e9acdc81d4a50b359e5e
*httpd-2.4.43.tar.gz

The SVN tag is '2.4.43' at r1875715.


+1 to release and thanks a bunch for RM!

Summary: all OK except for

- very few shutdown crashes on Solaris (already observed in 2.4.37, but 
then with MPM event when statically linked, now once with prefork and 
shared linking). Happens in mod_watchdog. Maybe prefork doesn't expect 
another thread running and doing deinit. gdb info at end.


- another shutdown crash on Solaris in mod_watchdog for prefork. gdb 
info at end.


- sporadic hangs with prefork plus mod_ext_filter on Linux (see separate 
thread).


Detailed report:

- Sigs and hashes OK
- contents of tarballs identical
- contents of tag and tarballs identical
  except for expected deltas

Built on

- Solaris 10 Sparc as 32 Bit Binaries
- SLES 11+12+15 (64 Bits)
- RHEL 6+7 (64 Bits)

For all platforms built

- with default (shared) and static modules
- with module set reallyall
- using --enable-load-all-modules
- against external APR/APU 1.7.0/1.6.1
  plus APR/APU 1.6.5/1.6.1
  plus APR/APU 1.7.x HEAD/1.7.x HEAD with expat
  plus APR/APU 1.7.x HEAD/1.7.x HEAD with libxml2
  plus APR/APU from deps tarball

- using external libraries
  - expat 2.2.9
  - pcre 8.44
  - lua 5.3.5 (compiled with LUA_COMPAT_MODULE)
  - libxml2 2.9.10
  - libnghttp2 1.40.0
  - brotli 1.0.7
  - curl 7.69.1
  - jansson 2.12
and
  - openssl 0.9.8zh, 1.0.2, 1.0.2u, 1.0.1e, 1.0.1l, 1.1.1, 1.1.1e plus 
patches (head of a few days ago)


- Tool chain:
- platform gcc except on Solaris
  (gcc 9.3.0 Solaris 10)
- CFLAGS: -O2 -g -Wall -fno-strict-aliasing
  - on Solaris additionally -mpcu=v9, -D_XOPEN_SOURCE,
-D_XOPEN_SOURCE_EXTENDED=1, -D__EXTENSIONS__
and -D_XPG6

All of the 660 builds succeeded, a few are still ongoing.

- compiler warnings: only on Solaris (GCC 9.3.0):
srclib/apr/locks/unix/proc_mutex.c:979:49: warning: 
'mutex_proc_pthread_cond_methods' defined but not used 
[-Wunused-const-variable=]


Tested for

- Solaris 10, SLES 11+12+15, RHEL 6+7
- MPMs prefork, worker, event
- default and static module builds
- log level trace8
- module set reallyall (127 modules)
- Perl client bundle build against OpenSSL 1.1.1, 1.1.0i, 1.0.2p and 0.9.8zh
- OpenSSL once linked statically and once as a shared library

Every OpenSSL version in the client tested with 1.1.1e plus patches in 
the server. Tests for more server OpenSSL versions are ongoing.


The total number of test suite runs was 680 (many more to come ...).

The following test failures were seen:

a Crashes only on Solaris, only with prefork MPM and
  dynamically linked builds.
  The crash seems to happen only at the end of a process during pchild
  clean up and it might be problematic, that the watchdog thread at that
  time still exists.
  gdb info see at end.

b Tests 4, 8 and 12 of t/modules/buffer.t
  Not a regression
  Tests 4, 8 and sometimes 12, always line 37
  Relatively frequent (93 times) failures on platforms Solaris 10
  and old SLES11 (114 times), RHEL 6 (88 times), but not on more modern
  (and here faster) SLES 12 and RHEL 7.
  Happens for all OpenSSL client and server
  versions and all link types.

c Test 5 in t/modules/dav.t line 69:
  Not a regression.
  22 times: twice RHEL 6, 3 times RHEL 7 and 5 times SLES 11,
  8 times SLES 12, twice SLES 15, twice Solaris 10.
  Creation, modified and now times not in the correct order.
  This seems to be a system issue, all tests done on NFS,
  many tested on virtualized guests.

d Tests 45, 48, 51, 54 in t/modules/cgi.t line 232:
  Not a regression
  125 times once Solaris
  Test checks log contents. Could be false positive due to
  logs written to NFS.

Regards,

Rainer

GDB info (sporadic) Solaris shutdown crashes during OpenSSL shutdown in 
mod_watchdog:


 fedd7668 realfree (1c0268, 61, 60, 1bfc58, 0, 1c02d8) + 154
 fedd7d9c _free_unlocked (feeb92b0, 0, d86dc, feeb9330, feeb03d8, 
1c99f8) + b0
 fedd7cd8 free (1c99f8, fe9a3ad0, d871c, fe8e4ee0, feeb03d8, 
feeb3a20) + 24
 fe8e2390 OPENSSL_LH_free (1bce10, fe9a3ad0, feeb5900, 2, fe9a3ad0, 
1cd450) + 64
 fe8bcf44 err_cleanup (0, f8800, feeb5900, fe9ed05c, fe8db4f0, 
fe9ecf4c) + 94
 fe8dee54 OPENSSL_cleanup (1, fe9ed284, fe9d3598, fe9a3240, fe9ed25c, 
fe9ed280) + 1e4

 fedc2374 

Re: Solaris, prefork, accept mutex and mod_ext_filter (Was: Prefork MPM and mod_watchdog)

2020-03-29 Thread Rainer Jung
I can now see the same problem on Linux (eg. RHEL 7, SLES 12 and SLES 
15) when doing testing for 2.4.43. I think it is not a regression and 
for me it is not a showstopper, but something that would be nice to fix. 
Symptom is


[Sat Mar 28 15:53:21.121461 2020] [mpm_prefork:debug] [pid 4574] 
prefork.c(919): AH00165: Accept mutex: pthread (default: pthread)
[Sat Mar 28 15:58:19.251858 2020] [mpm_prefork:emerg] [pid 6509] 
(22)Invalid argument: AH00144: couldn't grab the accept mutex
[Sat Mar 28 15:58:20.624995 2020] [mpm_prefork:emerg] [pid 4902] 
(22)Invalid argument: AH00146: couldn't release the accept mutex
[Sat Mar 28 15:58:21.410552 2020] [:emerg] [pid 4574] AH02818: MPM run 
failed, exiting


happening during t/modules/ext_filter.t and as a result all httpd 
processes are gone.


Although it happens only rarely, I think it does not happen when using 
APR 1.6, but only for 1.7 (1.7.0 and also for latest head). The accept 
mutex is pthread based. There are changes in APR 1.7 for those.


Since I only observe it for mod_ext_filter, there should be some 
dependency with the forked perl process.


I didn't yet have the opportunity to check, how much of the follwing 
description for Solaris also holds for Linux.


Regards,

Rainer

Am 03.02.2019 um 13:30 schrieb Rainer Jung:
I can now frequently reproduce running t/modules/ext_filter.t only. I 
stripped the reproducer down to the part of t/modules/ext_filter.t where 
it runs


POST "/apache/extfilter/out-limit/modules/cgi/perl_echo.pl", content => 
"foo and bar\n"


10 times in a row. If I increase the iteration count slightly to 20, the 
problem happens nearly every time. I also replaced perl_echo.pl and 
eval-cmd.pl by small C programs doing the echo and the s/foo/bar/ and 
can still reproduce.


This test involved mod_ext_filter and LimitRequestBody.

It seems I can not reproduce, if I shorten the POST body, so that it no 
longer hits the LimitRequestBody 6 configured for 
/apache/extfilter/out-limit/.


What happens, but I do not understand is:

- the first few requests are handled by two children in an alternating 
way. I can see the accept mutex being passed between these two children 
using lwp_mutex_timedlock(ADDRESS, 0x) and 
lwp_mutex_unlock(ADDRESS) using truss (Solaris variant of strace). So 
Solaris seems to implement the pthread mutexes via these lwp mutexes.


- after a few requests alternating between the two children, something 
strange happens:


   - child A returns from port_getn() for 22 (probably part of the 
accept() impl)

   - child A calls accept()
   - child A unlocks the accept mutex using lwp_mutex_unlock()
   - child B locks the accept mutex using lwp_mutex_timedlock()
   - child B calls port_associate for 22 (probably part of accept() impl)
   - child A handles the request
! - child A also calls port_associate for 22 (no sign of lock acquire)
! - child A returns from port_getn() for 22
   - child A calls accept() and starts to handle the connection
! - child B also returns from port_getn() for 22
! - child B gets EAGAIN for the accept()
   - child B calls port_associate for 22
   - child A handles the request
! - child A gets EDEADLK for pthread_mutex_lock() (this is from the 
error_log; there's no system call for this instance of 
pthread_mutex_lock() in the truss output). EDEADLK for 
pthread_mutex_lock() means that this process already has that lock.

   - child B returns from port_getn() for 22
   - child B calls accept() and starts to handle the connection
   - child A exits (now B is the only child left)
   - child B proceeds request handling
   - child B does all further port_associate(), port_getn(), and 
accept() plus connection and request handling. No more 
lwp_mutex_timedlock() or lwp_mutex_unlock() for B, maybe due to some 
optimization when only one process for a lock is left.



It is quite possible, that there is an underlying lwp_mutex or portfs 
bug here. But it is strange, that this only comes up when used with 
mod_ext_filter in combination with LimitRequestBody.


There was the fix

https://bz.apache.org/bugzilla/show_bug.cgi?id=60375

but I don't see, how this would influence the above list.

I can try to further narrow down (using static content to eliminate one 
forked child; using LimitRequestBody without mod_exit-filter etc.), but 
maybe someone already has an idea?


Regards,

Rainer


Re: OpenSSL 1.1.1e New EOF detection breaks session resumption

2020-03-27 Thread Rainer Jung

Am 27.03.2020 um 19:24 schrieb Steffen:


A discussion started on Apachelounge about an possible issue with 
OpenSSL 1.1.1e ( https://www.apachelounge.com/viewtopic.php?p=38941#38941 )


This is the introduced new EOF in 1.1.1e : 
https://github.com/openssl/openssl/commit/db943f43a60d1b5b1277e4b5317e8f288e7a0a3a 



Discussion on OpenSSL is at https://github.com/openssl/openssl/issues/11378

I dot understand what is going on, but  Daniel Stenberg (Curl) states 
:  The "poorly-implemented HTTP/1.1 servers" are still out there and are 
being used. How common? Impossible to say.



OpenSSL has a Patch with description :
... possible application breakage caused by a change in behavior 
introduced in 1.1.1e.  It affects at least nginx, which logs error 
messages such as:

nginx[16652]: [crit] 16675#0: *358 SSL_read() failed (SSL: error:
4095126:SSL routines:ssl3_read_n:unexpected eof while reading) while 
keepalive, client: , server: [::]:443


So looks  that nginx is effected.

My question is :
*Is Apache effected ? * Looks not, because till now: Apachelounge has 
more then a week 2.4.41 available with 1.1.1e, which is downloaded over 
50.000 times and no issues reported like this.


I did a few hundred test suite runs on 5 platforms for the 2.4.42 
release candidate against OpenSSL 1.1.1e and noticed no special new ssl 
related errors.


So either our tests do not detect it or httpd does not have that problem.

There will be a new OpenSSL 1.1.1f release next week.

Regards,

Rainer


Re: Broken: apache/httpd#515 (2.4.x - e936ddc)

2020-03-25 Thread Rainer Jung

Thanks to you for the help and patience anfd for maintaing Travis!

Regards,

Rainer

Am 25.03.2020 um 16:15 schrieb Joe Orton:

On Wed, Mar 25, 2020 at 11:20:32AM +0100, Rainer Jung wrote:

And now I notice that's of course not appropriate (everone needing to change
the call to the Makefile. So I will add local defaults in the spirit of your
below suggestion into Makefile.PL.


Thanks a lot, looks good now with r1875640!


Re: Errored: apache/httpd#521 (2.4.x - 5a6c1d7)

2020-03-25 Thread Rainer Jung

Only err'd for one platform:

Err:1 http://ports.ubuntu.com/ubuntu-ports bionic-updates/main arm64 
libnss-systemd arm64 237-3ubuntu10.39


  Temporary failure resolving 'ports.ubuntu.com'

so unrelated to our code and build/test files.

Regards,

Rainer

Am 25.03.2020 um 16:25 schrieb Travis CI:

apache

/

httpd

<https://travis-ci.org/github/apache/httpd?utm_medium=notification_source=email> 



branch icon2.4.x <https://github.com/apache/httpd/tree/2.4.x>

build has errored
Build #521 has errored 
<https://travis-ci.org/github/apache/httpd/builds/666832095?utm_medium=notification_source=email>

arrow to build time
clock icon11 mins and 11 secs

Rainer Jung avatarRainer Jung

5a6c1d7 CHANGESET → 
<https://github.com/apache/httpd/compare/a6c6191bea8a...5a6c1d7d53df>


Trivial change to trigger Travis build.


git-svn-id: 
https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1875646 
13f79535-47bb-0310-9956-ffa450edef68


Want to know about upcoming build environment updates?

Would you like to stay up-to-date with the upcoming Travis CI build 
environment updates? We set up a mailing list for you!


SIGN UP HERE <http://eepurl.com/9OCsP>

book icon

Documentation <https://docs.travis-ci.com/> about Travis CI

Have any questions? We're here to help. <mailto:supp...@travis-ci.com>
Unsubscribe 
<https://travis-ci.org/account/preferences/unsubscribe?repository=69847_medium=notification_source=email> 
from build emails from the apache/httpd repository.
To unsubscribe from *all* build emails, please update your settings 
<https://travis-ci.org/account/preferences/unsubscribe?utm_medium=notification_source=email>. 


black and white travis ci logo <https://travis-ci.com>

Travis CI GmbH, Rigaer Str. 8, 10427 Berlin, Germany | GF/CEO: Randy 
Jacops | Contact: cont...@travis-ci.com <mailto:cont...@travis-ci.com> | 
Amtsgericht Charlottenburg, Berlin, HRB 140133 B | Umsatzsteuer-ID gemäß 
§27 a Umsatzsteuergesetz: DE282002648


Re: Broken: apache/httpd#515 (2.4.x - e936ddc)

2020-03-25 Thread Rainer Jung

Am 25.03.2020 um 11:06 schrieb Joe Orton:

On Tue, Mar 24, 2020 at 11:35:38PM +0100, Rainer Jung wrote:

Excellent. That gave me the right idea where to look at.

I found a way to pass additionals args inside Makefile.PL into
Apache::TestMM::Argv which get automatically added to $vars in
Apache::TestConfig. It seems to work well. Committed in r1875598.


Doesn't seem to work for me or in Travis -

https://travis-ci.org/github/apache/httpd/jobs/06723#L2462

I don't up with with any Argv defined for limitrequestline if either
passing -apxs to Makefile.PL or not passing any argv.


Sorry, I forgot: we have to run the Makefile with

perl ./Makefile.PL -apxs foo -limitrequestline 128 -limitrequestlinex2 256

And now I notice that's of course not appropriate (everone needing to 
change the call to the Makefile. So I will add local defaults in the 
spirit of your below suggestion into Makefile.PL.


Regards,

Rainer


starting w/below patch applied:

sh-5.0$ perl ./Makefile.PL -apxs foo &> /dev/null
sh-5.0$ grep Argv t/TEST
$Apache::TestConfig::Argv{'limitrequestline'} = q|128|;
$Apache::TestConfig::Argv{'apxs'} = q|foo|;
$Apache::TestConfig::Argv{'limitrequestlinex2'} = q|256|;
sh-5.0$ svn revert Makefile.PL
Reverted 'Makefile.PL'
sh-5.0$ perl ./Makefile.PL -apxs foo &> /dev/null
sh-5.0$ grep Argv t/TEST
$Apache::TestConfig::Argv{'apxs'} = q|foo|;


Index: Makefile.PL
===
--- Makefile.PL (revision 1875618)
+++ Makefile.PL (working copy)
@@ -28,13 +28,12 @@
  # supported in an Apache::Test release.
  # Code borrowed from Apache::TestMM::filter_args().
  my %local_args = (
-limitrequestline => 'Value for LimitRequestLine',
-limitrequestlinex2 => 'Twice the value for LimitRequestLine',
+limitrequestline => '128',
+limitrequestlinex2 => '256',
  );
-my($argv, $vars) = Apache::TestConfig::filter_args(\@ARGV, \%local_args);
-@ARGV = @$argv;
-push(@Apache::TestMM::Argv, %$vars);
  
+push(@Apache::TestMM::Argv, %local_args);

+
  for my $script (@scripts) {
  Apache::TestMM::generate_script($script);
  }





Thanks and regards,

Rainer

Am 24.03.2020 um 18:35 schrieb Joe Ortqon:

On Tue, Mar 24, 2020 at 05:55:20PM +0100, Rainer Jung wrote:

I've got the following problem: I want to use a new config var in
Apache::Test as a patern to replace in extra.conf.in. I added the bvar to
Apache::Test::Config, but it seems Travis uses only a released version of
Apache::Test. Is there a way of influencing Apache::Test early from our own
scripts, so that I can set a default value for the new var?


This is the downside of relying on a released Apache::Test :(

If we can patch t/TEST I think we can set the defaults.  Not sure
if there is a clean way to do it but Makefile.PL is generating the file
so it's possible in theory.

If I apply this then ./t/TEST runs again with external Apache::Test

--- t/TEST~ 2020-03-12 11:46:26.823610447 +
+++ t/TEST  2020-03-24 17:31:43.225348563 +
@@ -9,6 +9,8 @@
   BEGIN { eval { require blib && blib->import; } }
   $Apache::TestConfig::Argv{'apxs'} = 
q|/home/jorton/src/asf/httpd-git/check/bin/apxs|;
+$Apache::TestConfig::Argv{'limitrequestline'} = q|128|;
+$Apache::TestConfig::Argv{'limitrequestlinex2'} = q|256|;
   use strict;
   use warnings FATAL => 'all';
@@ -19,4 +21,4 @@
   );
-use Apache::TestRun ();Apache::TestRun->new->run(@ARGV);
\ No newline at end of file
+use Apache::TestRun ();Apache::TestRun->new->run(@ARGV);







Error message:

[  error] configure() has failed:

invalid token: @limitrequestline@ in file
/home/travis/build/apache/httpd/test/perl-framework/t/conf/extra.conf.in

I introduces the new variable in extra.conf.in in r1875569 and the needed
code in Apache::TestConfig in r1875568.

Any help welcome. Otherwise I will revert and run with local modifications.

Thanks and regards,

Rainer

Am 24.03.2020 um 14:30 schrieb Travis CI:

apache

/

httpd

<https://travis-ci.org/github/apache/httpd?utm_medium=notification_source=email>


branch icon2.4.x <https://github.com/apache/httpd/tree/2.4.x>

build has failed
Build #515 was broken 
<https://travis-ci.org/github/apache/httpd/builds/666326658?utm_medium=notification_source=email>
arrow to build time
clock icon9 mins and 52 secs

Jim Jagielski avatarJim Jagielski

e936ddc CHANGESET →
<https://github.com/apache/httpd/compare/5855f218dcf9...e936ddc9ce2c>

2.4.42 was DOA


git-svn-id:
https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1875576
13f79535-47bb-0310-9956-ffa450edef68

Want to know about upcoming build environment updates?

Would you like to stay up-to-date with the upcoming Travis CI build
environment updates? We set up a mailing list for you!

SIGN UP HERE <http://eepurl.com/9OCsP>

book icon

Documentation <https://docs.travis-ci.com/> about Travis CI

Have any questions? We're here to help. <mailto:supp...@tr

Re: Broken: apache/httpd#515 (2.4.x - e936ddc)

2020-03-24 Thread Rainer Jung

Excellent. That gave me the right idea where to look at.

I found a way to pass additionals args inside Makefile.PL into 
Apache::TestMM::Argv which get automatically added to $vars in 
Apache::TestConfig. It seems to work well. Committed in r1875598.


Thanks and regards,

Rainer

Am 24.03.2020 um 18:35 schrieb Joe Ortqon:

On Tue, Mar 24, 2020 at 05:55:20PM +0100, Rainer Jung wrote:

I've got the following problem: I want to use a new config var in
Apache::Test as a patern to replace in extra.conf.in. I added the bvar to
Apache::Test::Config, but it seems Travis uses only a released version of
Apache::Test. Is there a way of influencing Apache::Test early from our own
scripts, so that I can set a default value for the new var?


This is the downside of relying on a released Apache::Test :(

If we can patch t/TEST I think we can set the defaults.  Not sure
if there is a clean way to do it but Makefile.PL is generating the file
so it's possible in theory.

If I apply this then ./t/TEST runs again with external Apache::Test

--- t/TEST~ 2020-03-12 11:46:26.823610447 +
+++ t/TEST  2020-03-24 17:31:43.225348563 +
@@ -9,6 +9,8 @@
  BEGIN { eval { require blib && blib->import; } }
  
  $Apache::TestConfig::Argv{'apxs'} = q|/home/jorton/src/asf/httpd-git/check/bin/apxs|;

+$Apache::TestConfig::Argv{'limitrequestline'} = q|128|;
+$Apache::TestConfig::Argv{'limitrequestlinex2'} = q|256|;
  
  use strict;

  use warnings FATAL => 'all';
@@ -19,4 +21,4 @@
  );
  
  
-use Apache::TestRun ();Apache::TestRun->new->run(@ARGV);

\ No newline at end of file
+use Apache::TestRun ();Apache::TestRun->new->run(@ARGV);







Error message:

[  error] configure() has failed:

invalid token: @limitrequestline@ in file
/home/travis/build/apache/httpd/test/perl-framework/t/conf/extra.conf.in

I introduces the new variable in extra.conf.in in r1875569 and the needed
code in Apache::TestConfig in r1875568.

Any help welcome. Otherwise I will revert and run with local modifications.

Thanks and regards,

Rainer

Am 24.03.2020 um 14:30 schrieb Travis CI:

apache

/

httpd

<https://travis-ci.org/github/apache/httpd?utm_medium=notification_source=email>


branch icon2.4.x <https://github.com/apache/httpd/tree/2.4.x>

build has failed
Build #515 was broken 
<https://travis-ci.org/github/apache/httpd/builds/666326658?utm_medium=notification_source=email>
arrow to build time
clock icon9 mins and 52 secs

Jim Jagielski avatarJim Jagielski

e936ddc CHANGESET →
<https://github.com/apache/httpd/compare/5855f218dcf9...e936ddc9ce2c>

2.4.42 was DOA


git-svn-id:
https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1875576
13f79535-47bb-0310-9956-ffa450edef68

Want to know about upcoming build environment updates?

Would you like to stay up-to-date with the upcoming Travis CI build
environment updates? We set up a mailing list for you!

SIGN UP HERE <http://eepurl.com/9OCsP>

book icon

Documentation <https://docs.travis-ci.com/> about Travis CI

Have any questions? We're here to help. <mailto:supp...@travis-ci.com>
Unsubscribe 
<https://travis-ci.org/account/preferences/unsubscribe?repository=69847_medium=notification_source=email>
from build emails from the apache/httpd repository.
To unsubscribe from *all* build emails, please update your settings 
<https://travis-ci.org/account/preferences/unsubscribe?utm_medium=notification_source=email>.

black and white travis ci logo <https://travis-ci.com>

Travis CI GmbH, Rigaer Str. 8, 10427 Berlin, Germany | GF/CEO: Randy
Jacops | Contact: cont...@travis-ci.com <mailto:cont...@travis-ci.com> |
Amtsgericht Charlottenburg, Berlin, HRB 140133 B | Umsatzsteuer-ID gemäß
§27 a Umsatzsteuergesetz: DE282002648


Re: Broken: apache/httpd#515 (2.4.x - e936ddc)

2020-03-24 Thread Rainer Jung
I've got the following problem: I want to use a new config var in 
Apache::Test as a patern to replace in extra.conf.in. I added the bvar 
to Apache::Test::Config, but it seems Travis uses only a released 
version of Apache::Test. Is there a way of influencing Apache::Test 
early from our own scripts, so that I can set a default value for the 
new var?


Error message:

[  error] configure() has failed:

invalid token: @limitrequestline@ in file 
/home/travis/build/apache/httpd/test/perl-framework/t/conf/extra.conf.in


I introduces the new variable in extra.conf.in in r1875569 and the 
needed code in Apache::TestConfig in r1875568.


Any help welcome. Otherwise I will revert and run with local modifications.

Thanks and regards,

Rainer

Am 24.03.2020 um 14:30 schrieb Travis CI:

apache

/

httpd

 



branch icon2.4.x 

build has failed
Build #515 was broken 


arrow to build time
clock icon9 mins and 52 secs

Jim Jagielski avatarJim Jagielski

e936ddc CHANGESET → 



2.4.42 was DOA


git-svn-id: 
https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1875576 
13f79535-47bb-0310-9956-ffa450edef68


Want to know about upcoming build environment updates?

Would you like to stay up-to-date with the upcoming Travis CI build 
environment updates? We set up a mailing list for you!


SIGN UP HERE 

book icon

Documentation  about Travis CI

Have any questions? We're here to help. 
Unsubscribe 
 
from build emails from the apache/httpd repository.
To unsubscribe from *all* build emails, please update your settings 
. 


black and white travis ci logo 

Travis CI GmbH, Rigaer Str. 8, 10427 Berlin, Germany | GF/CEO: Randy 
Jacops | Contact: cont...@travis-ci.com  | 
Amtsgericht Charlottenburg, Berlin, HRB 140133 B | Umsatzsteuer-ID gemäß 
§27 a Umsatzsteuergesetz: DE282002648


Re: svn commit: r1875569 - in /httpd/test/framework/trunk/t: apache/limits.t conf/extra.conf.in

2020-03-24 Thread Rainer Jung

Hi Yann,

Am 24.03.2020 um 14:04 schrieb Yann Ylavic:

On Tue, Mar 24, 2020 at 11:13 AM  wrote:


Author: rjung
Date: Tue Mar 24 10:13:36 2020
New Revision: 1875569

[]


+my $limitrequestlinex2 = Apache::Test::config()->{vars}->{limitrequestlinex2};


s/linex2};/line} * 2;/ ?


The x2 variant is also used in

t/conf/extra.conf.in

RewriteCond expr "util_strlen(%{REQUEST_FILENAME}) -le @limitrequestlinex2@"

and since our expr doesn't allow arithmetic expressions (apart from 
comparison) I had to introduce the x2 variant. So once it is there I 
though it would be consistent to use it in both places. But of course, 
we could remove its use in the perl script. Do you have a strong preference?


Regards,

Rainer


Re: Use of [skip ci] in commit messages to avoid Travis builds

2020-03-23 Thread Rainer Jung

Am 23.03.2020 um 20:12 schrieb Ruediger Pluem:



On 3/23/20 6:22 PM, Rainer Jung wrote:

Am 23.03.2020 um 16:56 schrieb Joe Orton:

On Sat, Feb 08, 2020 at 12:01:29PM +0100, Luca Toscano wrote:

Hi everybody,

Travis is able to read commit messages and skip the CI workflow if the
"[skip ci]" magic sequence is added somewhere. I keep forgetting about
it too, but it would be nice if we could start using it to avoid using
CI resources/workers when not needed (like docs changes, entries in
STATUS, etc..). I didn't find a way to instruct Travis to avoid
triggering a build if only certain file types are committed, so the
only solution for the moment is to manually add the aforementioned
sequence :(

It is not a big deal to trigger builds even for docs etc.., but the
ASF resources/workers are limited (and shared among projects IIUC) so
I am only suggesting to be good neighbors :) If this is totally insane
or unacceptable I'll back off and stop vouching for it I promise!


I found a new option here while playing with conditionals, we can skip
an entire build based on the commit message, it appears -

https://github.com/apache/httpd/pull/87/commits/7dd995c555ac0aea6cb3d58634750e2e076eb0cc

so this could catch a lot of the quite pointless Travis runs for STATUS
changes by filtering by commit message, something like...

if: (branch != "2.4.x" and commit_message !~ /^[Tt]ransforms$/) or
  (branch = "2.4.x" and commit_message !~ 
/^([Pp]ropose|[Vv]ote|[Tt]ransforms)$/)

anything else??


Since I forgot the [skip ci] today, I too a little time to inspect svn log for 
STATUS and arrived at the following, probably not
maintainable regexp (perl notation) for 2.4.x:

/^i?(\* ?)?([0-9] *)?([sS]ome +|[mM]ore +|[aA]dd( a)? +|[eE]asy +|[qQ]uick 
+|[sS]mall +|[uU]pdate( after)? +)?([bB]ackport(ed)?
+)?([pP]ropos(e|als?),?( *((and)?|\/|\+) *([vV]ote)?)?|[vV]otes?,?( 
*((and)?|\/|\+)
*([pP]ropos(e|als?)|[pP]romot(e|ion)s?|[cC]omments?))?|[pP]romot(e|ion)s?|[cC]omment|[nN]ote|[dD]one|[mM]erged|[cC]ommitted|[bB]ackport(ed)?)[\.:!]?(
*\[skip ci\])?$/


I think the backport stuff should not be in the list as it is often used in 
backport commits which I want to see run afterwards
through the tests for that branch.


Not sure how a multi-line commit message will be handled, ie. if the 
caret and dollar anchors are applied to every line and what happens if 
one line our of many matches.


The above pattern was meant to be applied to one-line commit messages 
only or phrased differently it should match the whole commit message, 
not only individual lines. In that case I would expect the backport part 
of the pattern to be no problem. A real backport commit message should 
also contain info like the revision number or the original commit 
message and would thus not match.


But I don't know whether the assumption about multi-line commit message 
matching is correct.


Regards,

Rainer


Re: Use of [skip ci] in commit messages to avoid Travis builds

2020-03-23 Thread Rainer Jung

Am 23.03.2020 um 16:56 schrieb Joe Orton:

On Sat, Feb 08, 2020 at 12:01:29PM +0100, Luca Toscano wrote:

Hi everybody,

Travis is able to read commit messages and skip the CI workflow if the
"[skip ci]" magic sequence is added somewhere. I keep forgetting about
it too, but it would be nice if we could start using it to avoid using
CI resources/workers when not needed (like docs changes, entries in
STATUS, etc..). I didn't find a way to instruct Travis to avoid
triggering a build if only certain file types are committed, so the
only solution for the moment is to manually add the aforementioned
sequence :(

It is not a big deal to trigger builds even for docs etc.., but the
ASF resources/workers are limited (and shared among projects IIUC) so
I am only suggesting to be good neighbors :) If this is totally insane
or unacceptable I'll back off and stop vouching for it I promise!


I found a new option here while playing with conditionals, we can skip
an entire build based on the commit message, it appears -

https://github.com/apache/httpd/pull/87/commits/7dd995c555ac0aea6cb3d58634750e2e076eb0cc

so this could catch a lot of the quite pointless Travis runs for STATUS
changes by filtering by commit message, something like...

if: (branch != "2.4.x" and commit_message !~ /^[Tt]ransforms$/) or
 (branch = "2.4.x" and commit_message !~ 
/^([Pp]ropose|[Vv]ote|[Tt]ransforms)$/)

anything else??


Since I forgot the [skip ci] today, I too a little time to inspect svn 
log for STATUS and arrived at the following, probably not maintainable 
regexp (perl notation) for 2.4.x:


/^i?(\* ?)?([0-9] *)?([sS]ome +|[mM]ore +|[aA]dd( a)? +|[eE]asy 
+|[qQ]uick +|[sS]mall +|[uU]pdate( after)? +)?([bB]ackport(ed)? 
+)?([pP]ropos(e|als?),?( *((and)?|\/|\+) *([vV]ote)?)?|[vV]otes?,?( 
*((and)?|\/|\+) 
*([pP]ropos(e|als?)|[pP]romot(e|ion)s?|[cC]omments?))?|[pP]romot(e|ion)s?|[cC]omment|[nN]ote|[dD]one|[mM]erged|[cC]ommitted|[bB]ackport(ed)?)[\.:!]?( 
*\[skip ci\])?$/


It is roughly like yours, but allows a few prefixes and suffixes around 
the magic words plus a few more of them and some frequent combinations.


Regards,

Rainer


Re: [VOTE] Release httpd-2.4.42

2020-03-23 Thread Rainer Jung

Am 23.03.2020 um 16:18 schrieb Daniel Ruggeri:

Hi, all;
    Per the issues surfaced/fixed, I'll go ahead and declare this release
as dead-on-the vine. I'll target another T later this week, hopefully
after the discussion around OpenSSL versioning plays out.

    How about Thursday?


Works for me.
Thanks!
Rainer


Re: svn commit: r1875544 - /httpd/httpd/trunk/modules/ssl/ssl_engine_kernel.c

2020-03-23 Thread Rainer Jung

Thanks for the feedback. Proposed for 2.4.x a minute ago.

Am 23.03.2020 um 14:48 schrieb Ruediger Pluem:



On 3/23/20 2:44 PM, Rainer Jung wrote:

The dependency on SSL_CTX_get_min_proto_version() and 
SSL_CTX_get_max_proto_version() was introduced in October by Yann's
"r1868645 mod_ssl: negotiate the TLS protocol version per name based vhost 
configuration".

Although the set variants are available in 1.1.0, the set were added later in 
1.1.0g.

Not sure, whether adjusting the version check as done now is the right fix. At 
least it unbreaks building httpd against OpenSSL
1.1.0-1.1.0f.

The original change has been backported to 2.4.x, so building that for the 
above OpenSSL versions is currently broken.


IMHO we should backport it then once clarified that this is the correct thing 
to do and ensure that it gets in 2.4.43.
I think this is a release blocker.

Regards

Rüdiger


Re: svn commit: r1875544 - /httpd/httpd/trunk/modules/ssl/ssl_engine_kernel.c

2020-03-23 Thread Rainer Jung
The dependency on SSL_CTX_get_min_proto_version() and 
SSL_CTX_get_max_proto_version() was introduced in October by Yann's 
"r1868645 mod_ssl: negotiate the TLS protocol version per name based 
vhost configuration".


Although the set variants are available in 1.1.0, the set were added 
later in 1.1.0g.


Not sure, whether adjusting the version check as done now is the right 
fix. At least it unbreaks building httpd against OpenSSL 1.1.0-1.1.0f.


The original change has been backported to 2.4.x, so building that for 
the above OpenSSL versions is currently broken.


Regards,

Rainer

Am 23.03.2020 um 14:33 schrieb rj...@apache.org:

Author: rjung
Date: Mon Mar 23 13:33:22 2020
New Revision: 1875544

URL: http://svn.apache.org/viewvc?rev=1875544=rev
Log:
Fix compilation breakage with OpenSSL 1.1.0 up to 1.1.0f.
SSL_CTX_get_min_proto_version() and
SSL_CTX_get_max_proto_version() were only introduced in
1.1.0g.

Modified:
 httpd/httpd/trunk/modules/ssl/ssl_engine_kernel.c

Modified: httpd/httpd/trunk/modules/ssl/ssl_engine_kernel.c
URL: 
http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/ssl/ssl_engine_kernel.c?rev=1875544=1875543=1875544=diff
==
--- httpd/httpd/trunk/modules/ssl/ssl_engine_kernel.c (original)
+++ httpd/httpd/trunk/modules/ssl/ssl_engine_kernel.c Mon Mar 23 13:33:22 2020
@@ -2535,7 +2535,7 @@ static int ssl_find_vhost(void *serverna
   * from the ctx by hand
   */
  SSL_set_options(ssl, SSL_CTX_get_options(ctx));
-#if OPENSSL_VERSION_NUMBER >= 0x1010L \
+#if OPENSSL_VERSION_NUMBER >= 0x1010007fL \
  && (!defined(LIBRESSL_VERSION_NUMBER) \
  || LIBRESSL_VERSION_NUMBER >= 0x2080L)
  /*




--
kippdata
informationstechnologie GmbH   Tel: 0228 98549 -0
Bornheimer Str. 33aFax: 0228 98549 -50
53111 Bonn www.kippdata.de

HRB 8018 Amtsgericht Bonn / USt.-IdNr. DE 196 457 417
Geschäftsführer: Dr. Thomas Höfer, Rainer Jung, Sven Maurmann


mod_proxy_ajp backport for "secret" attribute to 2.4.x

2020-02-23 Thread Rainer Jung
Just a heads up: the support for the "secret" atribute in mod_proxy_ajp 
has not been backported:


https://bz.apache.org/bugzilla/show_bug.cgi?id=53098

Tomcat hardened its AJP connector in the latest patch releases and by 
default now requires the proxy to send such a "secret". This can be 
turned off but is not recommended.


I think we should backport r1738878 plus small struct layout adjustments 
for compatibility in 2.4.x.


I could not yet test it, but the diff seems to apply well apart from 
struct layout, which we need to trivially adjust anyays (move new 
members to end of struct).


If anyone would be able to test and propose before I get to it, that 
would be great.


Regards,

Rainer


Re: svn commit: r1860166 - in /httpd/httpd/branches/2.4.x: ./ docs/manual/mod/ include/ modules/http2/ modules/proxy/ server/

2019-10-29 Thread Rainer Jung
Double check: the condition in the do-while loop that was chaned to a 
while loop has also changed:


FROM

do { ... }
while ((bytes_read < MAX_MEM_SPOOL - 80)
   && !APR_BUCKET_IS_EOS(APR_BRIGADE_LAST(input_brigade))
   && !req->prefetch_nonblocking);

TO

while (((bytes_read < MAX_MEM_SPOOL - 80)
   && (APR_BRIGADE_EMPTY(input_brigade)
   || !APR_BUCKET_IS_EOS(APR_BRIGADE_LAST(input_brigade 
{ ... }


That's intended?

Regards,

Rainer

Am 29.10.2019 um 16:23 schrieb Rainer Jung:

Am 29.10.2019 um 16:19 schrieb Yann Ylavic:

Hi Rainer,

thanks for looking at this.



Aha, and this is due to the fact, that r1656259 "mod_proxy_http: don't
connect or reuse backend before prefetching request body." or parts of
it was backported from trunk to 2.4 as part of r1860166.


Yes, that's where prefetch was moved before connect, but we shouldn't
change that IMHO, besides fixing bugs...

Let me look at how we can reuse the input_brigade between balancers. I
have a small patch already and testing it.



So I think (not yet verified), that the same problems applies to trunk
since r1656259 in February 2015 :(


Yes, I'm fixing on trunk (first).


Thank you Yann. Let me know when I should test something. It's OK, if it 
is not yet the final fix ;)


Regards,

Rainer


Re: svn commit: r1860166 - in /httpd/httpd/branches/2.4.x: ./ docs/manual/mod/ include/ modules/http2/ modules/proxy/ server/

2019-10-29 Thread Rainer Jung

Hi Yann,

Am 29.10.2019 um 16:58 schrieb Yann Ylavic:

On Tue, Oct 29, 2019 at 4:24 PM Rainer Jung  wrote:


Thank you Yann. Let me know when I should test something. It's OK, if it
is not yet the final fix ;)


The attached patch seems to work for me..


LGTM. I applied/ported to 2.4.x, it fixes the problem for me as well and 
looks local enough to not intruduce new problems.


Thanks!

Rainer


Re: svn commit: r1860166 - in /httpd/httpd/branches/2.4.x: ./ docs/manual/mod/ include/ modules/http2/ modules/proxy/ server/

2019-10-29 Thread Rainer Jung

Am 29.10.2019 um 16:19 schrieb Yann Ylavic:

Hi Rainer,

thanks for looking at this.



Aha, and this is due to the fact, that r1656259 "mod_proxy_http: don't
connect or reuse backend before prefetching request body." or parts of
it was backported from trunk to 2.4 as part of r1860166.


Yes, that's where prefetch was moved before connect, but we shouldn't
change that IMHO, besides fixing bugs...

Let me look at how we can reuse the input_brigade between balancers. I
have a small patch already and testing it.



So I think (not yet verified), that the same problems applies to trunk
since r1656259 in February 2015 :(


Yes, I'm fixing on trunk (first).


Thank you Yann. Let me know when I should test something. It's OK, if it 
is not yet the final fix ;)


Regards,

Rainer



Re: svn commit: r1860166 - in /httpd/httpd/branches/2.4.x: ./ docs/manual/mod/ include/ modules/http2/ modules/proxy/ server/

2019-10-29 Thread Rainer Jung
Aha, and this is due to the fact, that r1656259 "mod_proxy_http: don't 
connect or reuse backend before prefetching request body." or parts of 
it was backported from trunk to 2.4 as part of r1860166.


So I think (not yet verified), that the same problems applies to trunk 
since r1656259 in February 2015 :(


Regards,

Rainer

Am 29.10.2019 um 14:21 schrieb Rainer Jung:
The reason why this fails now is that we prefetch in 2.4.41 the request 
body before doing the connection check to the backend. In 2.4.39 we did 
that after doing the check, so the body was still there when doing the 
final request sending.


2.4.39:

In proxy_http_handler():
- ap_proxy_determine_connection()
- ap_proxy_check_connection()
- optionally ap_proxy_connect_backend() which might fail.
- ap_proxy_connection_create_ex()
- ap_proxy_http_request() which does prefetch late!

2.4.41:

In proxy_http_handler():
- ap_proxy_determine_connection()
- early ap_proxy_http_prefetch() which does the prefetch!
- optionally again ap_proxy_determine_connection()
- ap_proxy_check_connection()
- optionally ap_proxy_connect_backend() which might fail.
- ap_proxy_connection_create_ex()
- ap_proxy_http_request()

So we either need to remember the prefetch result for later retries or 
switch back to the old order.


Regards,

Rainer

Am 29.10.2019 um 12:46 schrieb Rainer Jung:
This happens in the case of a small body. We read the body into 
req->input_brigade in ap_proxy_http_prefetch() before trying the first 
node, but then loose it on the second node, because we use another req 
and thus also another req->input_brigade then.


Not sure, how we could best save the read input_brigade for the second 
attempt after failover. Will try some experiments.


If you try to reproduce yourself, make sure you use a small POST 
(here: 30 bytes) and also exclude /favicon.ico from forwarding when 
using a browser. Otherwise some of the failovers will be triggered by 
favicon.ico and you'll not notice the problem in the POST request:


ProxyPass /favicon.ico !

Regards,

Rainer

Am 29.10.2019 um 11:15 schrieb Rainer Jung:
A first heads-up: it seems this commit broke failover for POST 
requests. Most (or all?) of the times a balancer failover happens for 
a POST request, the request send to the failover node has a 
Content-Length of "0" instead of the real content length.


I use a trivial setup like this:


   ProxySet lbmethod=byrequests
   BalancerMember http://localhost:5680
   BalancerMember http://localhost:5681


ProxyPass / balancer://backends/

where one backend node is up and the second node is down.

I will investigate further.

Regards,

Rainer


--
kippdata
informationstechnologie GmbH   Tel: 0228 98549 -0
Bornheimer Str. 33aFax: 0228 98549 -50
53111 Bonn www.kippdata.de

HRB 8018 Amtsgericht Bonn / USt.-IdNr. DE 196 457 417
Geschäftsführer: Dr. Thomas Höfer, Rainer Jung, Sven Maurmann


Re: svn commit: r1860166 - in /httpd/httpd/branches/2.4.x: ./ docs/manual/mod/ include/ modules/http2/ modules/proxy/ server/

2019-10-29 Thread Rainer Jung
The reason why this fails now is that we prefetch in 2.4.41 the request 
body before doing the connection check to the backend. In 2.4.39 we did 
that after doing the check, so the body was still there when doing the 
final request sending.


2.4.39:

In proxy_http_handler():
- ap_proxy_determine_connection()
- ap_proxy_check_connection()
- optionally ap_proxy_connect_backend() which might fail.
- ap_proxy_connection_create_ex()
- ap_proxy_http_request() which does prefetch late!

2.4.41:

In proxy_http_handler():
- ap_proxy_determine_connection()
- early ap_proxy_http_prefetch() which does the prefetch!
- optionally again ap_proxy_determine_connection()
- ap_proxy_check_connection()
- optionally ap_proxy_connect_backend() which might fail.
- ap_proxy_connection_create_ex()
- ap_proxy_http_request()

So we either need to remember the prefetch result for later retries or 
switch back to the old order.


Regards,

Rainer

Am 29.10.2019 um 12:46 schrieb Rainer Jung:
This happens in the case of a small body. We read the body into 
req->input_brigade in ap_proxy_http_prefetch() before trying the first 
node, but then loose it on the second node, because we use another req 
and thus also another req->input_brigade then.


Not sure, how we could best save the read input_brigade for the second 
attempt after failover. Will try some experiments.


If you try to reproduce yourself, make sure you use a small POST (here: 
30 bytes) and also exclude /favicon.ico from forwarding when using a 
browser. Otherwise some of the failovers will be triggered by 
favicon.ico and you'll not notice the problem in the POST request:


ProxyPass /favicon.ico !

Regards,

Rainer

Am 29.10.2019 um 11:15 schrieb Rainer Jung:
A first heads-up: it seems this commit broke failover for POST 
requests. Most (or all?) of the times a balancer failover happens for 
a POST request, the request send to the failover node has a 
Content-Length of "0" instead of the real content length.


I use a trivial setup like this:


   ProxySet lbmethod=byrequests
   BalancerMember http://localhost:5680
   BalancerMember http://localhost:5681


ProxyPass / balancer://backends/

where one backend node is up and the second node is down.

I will investigate further.

Regards,

Rainer


Re: svn commit: r1860166 - in /httpd/httpd/branches/2.4.x: ./ docs/manual/mod/ include/ modules/http2/ modules/proxy/ server/

2019-10-29 Thread Rainer Jung
This happens in the case of a small body. We read the body into 
req->input_brigade in ap_proxy_http_prefetch() before trying the first 
node, but then loose it on the second node, because we use another req 
and thus also another req->input_brigade then.


Not sure, how we could best save the read input_brigade for the second 
attempt after failover. Will try some experiments.


If you try to reproduce yourself, make sure you use a small POST (here: 
30 bytes) and also exclude /favicon.ico from forwarding when using a 
browser. Otherwise some of the failovers will be triggered by 
favicon.ico and you'll not notice the problem in the POST request:


ProxyPass /favicon.ico !

Regards,

Rainer

Am 29.10.2019 um 11:15 schrieb Rainer Jung:
A first heads-up: it seems this commit broke failover for POST requests. 
Most (or all?) of the times a balancer failover happens for a POST 
request, the request send to the failover node has a Content-Length of 
"0" instead of the real content length.


I use a trivial setup like this:


   ProxySet lbmethod=byrequests
   BalancerMember http://localhost:5680
   BalancerMember http://localhost:5681


ProxyPass / balancer://backends/

where one backend node is up and the second node is down.

I will investigate further.

Regards,

Rainer


Re: svn commit: r1860166 - in /httpd/httpd/branches/2.4.x: ./ docs/manual/mod/ include/ modules/http2/ modules/proxy/ server/

2019-10-29 Thread Rainer Jung
A first heads-up: it seems this commit broke failover for POST requests. 
Most (or all?) of the times a balancer failover happens for a POST 
request, the request send to the failover node has a Content-Length of 
"0" instead of the real content length.


I use a trivial setup like this:


  ProxySet lbmethod=byrequests
  BalancerMember http://localhost:5680
  BalancerMember http://localhost:5681


ProxyPass / balancer://backends/

where one backend node is up and the second node is down.

I will investigate further.

Regards,

Rainer

Am 28.05.2019 um 01:19 schrieb minf...@apache.org:

Author: minfrin
Date: Mon May 27 23:19:12 2019
New Revision: 1860166

URL: http://svn.apache.org/viewvc?rev=1860166=rev
Log:
mod_proxy_http: forward 100-continue, and minimize race conditions when
reusing backend connections. PR 60330.

+1: ylavic, icing, jim
ylavic: plus http://svn.apache.org/r1856036 (opt-out)
2.4.x patch: 
http://people.apache.org/~ylavic/patches/httpd-2.4.x-forward_100_continue-v6.patch
+1: ylavic, jim, minfrin

Modified:
 httpd/httpd/branches/2.4.x/CHANGES
 httpd/httpd/branches/2.4.x/STATUS
 httpd/httpd/branches/2.4.x/docs/manual/mod/mod_proxy.xml
 httpd/httpd/branches/2.4.x/include/ap_mmn.h
 httpd/httpd/branches/2.4.x/modules/http2/mod_proxy_http2.c
 httpd/httpd/branches/2.4.x/modules/proxy/mod_proxy.c
 httpd/httpd/branches/2.4.x/modules/proxy/mod_proxy.h
 httpd/httpd/branches/2.4.x/modules/proxy/mod_proxy_http.c
 httpd/httpd/branches/2.4.x/modules/proxy/mod_proxy_wstunnel.c
 httpd/httpd/branches/2.4.x/modules/proxy/proxy_util.c
 httpd/httpd/branches/2.4.x/server/protocol.c

Modified: httpd/httpd/branches/2.4.x/CHANGES
URL: 
http://svn.apache.org/viewvc/httpd/httpd/branches/2.4.x/CHANGES?rev=1860166=1860165=1860166=diff
==
--- httpd/httpd/branches/2.4.x/CHANGES [utf-8] (original)
+++ httpd/httpd/branches/2.4.x/CHANGES [utf-8] Mon May 27 23:19:12 2019
@@ -1,6 +1,9 @@
   -*- coding: utf-8 -*-
  Changes with Apache 2.4.40
  
+  *) mod_proxy_http: forward 100-continue, and minimize race conditions when

+ reusing backend connections. PR 60330. [Yann Ylavic, Jean-Frederic Clere]
+
*) Easy patches: synch 2.4.x and trunk
  - core: 80 chars
  - http_core: Clean-uo and style. No functional change overall

Modified: httpd/httpd/branches/2.4.x/STATUS
URL: 
http://svn.apache.org/viewvc/httpd/httpd/branches/2.4.x/STATUS?rev=1860166=1860165=1860166=diff
==
--- httpd/httpd/branches/2.4.x/STATUS (original)
+++ httpd/httpd/branches/2.4.x/STATUS Mon May 27 23:19:12 2019
@@ -128,41 +128,6 @@ PATCHES ACCEPTED TO BACKPORT FROM TRUNK:
[ start all new proposals below, under PATCHES PROPOSED. ]
  
  
-  *) mod_proxy_http: forward 100-continue, and minimize race conditions when

- reusing backend connections. PR 60330.
- trunk patch: http://svn.apache.org/r1656259
-  http://svn.apache.org/r1656359
-  http://svn.apache.org/r1721759
-  http://svn.apache.org/r1729507
-  http://svn.apache.org/r1729749
-  http://svn.apache.org/r1754159
-  http://svn.apache.org/r1836588
-  http://svn.apache.org/r1836648
-  http://svn.apache.org/r1836716
-  http://svn.apache.org/r1836750
-  http://svn.apache.org/r1837040
-  http://svn.apache.org/r1853407
-  http://svn.apache.org/r1853518
-  http://svn.apache.org/r1853561
-  http://svn.apache.org/r1853566
-  http://svn.apache.org/r1853953
-  http://svn.apache.org/r1853956
- 2.4.x patch: 
http://people.apache.org/~ylavic/patches/httpd-2.4.x-forward_100_continue-v4.patch
- ylavic: please note the "s/ASCII_(CRLF|ZERO)/\1_ASCII/g" in the backport,
- same as in trunk (where definitions come from httpd.h) so doing
- the change now in 2.4.x helps backports. Since in 2.4.x these
- macros are still locally #defined, I also added the #ifdefs around
- them to avoid potential issues..
- @icing: sorry I had to reset your vote for v4, but it looks
- sensible to me to have trunk and 2.4.x code in sync as much as
- possible. Changes from v3 to v4 (r1853953 mainly, r1853956 is only
- comment) are non functional (or at least intended as such).
- +1: ylavic, icing, jim
- ylavic: plus http://svn.apache.org/r1856036 (opt-out)
- 2.4.x patch: 
http://people.apache.org/~ylavic/patches/httpd-2.4.x-forward_100_continue-v6.patch
- +1: ylavic, jim, minfrin
-
-
  PATCHES PROPOSED TO BACKPORT FROM TRUNK:
[ New proposals should be added at the end of the list ]
  



Re: Time for httpd 2.6.x?

2019-10-23 Thread Rainer Jung

I guess you would want to take trunk as a starting point?

We would probably start with releasing (more) 2.5 alphas/betas. Maybe we 
can defer branching to the point, where we aim at the first 2.6.0 GA. At 
least I am not aware of a pressing need to do more breaking changes in 
trunk right now (and before we might reach 2.6.0). Just want to avoid 
that we branch unnecessarily early and have to do more backports than 
needed.


Anyways: I am +1 for aiming at 2.6.0 and when to branch is more of a 
procedural optimization.


Regards,

Rainer

Am 23.10.2019 um 08:28 schrieb Luca Toscano:

Not even a comment? :)

Luca

Il giorno dom 13 ott 2019 alle ore 20:58 Luca Toscano
 ha scritto:


Hi everybody,

in trunk's STATUS there are a lot of suggestions about things to
improve/change for 2.6.x. There have been discussions during the past
couple of years about how/when/if to create a 2.6 release branch, but
for a lot of reasons we didn't do any progress. Would it be something
to consider for the next months?

Luca


Re: Problem w/ Revision 1864435

2019-09-19 Thread Rainer Jung

Hi Bill,

Am 19.09.2019 um 19:39 schrieb William A Rowe Jr:

This commit somehow missed my inbox (and wasn't quoted in your observations)

http://svn.apache.org/viewvc?view=revision=1864435

Rainer, you observed in the commit notes;

The GCC flag "-Wno-error=comment" introduced byr1855446  

andr1850745    
are only known since GCC 4.2. Since it gets
set unconditionally, this breaks compilation with old GCC
even when not using maintainer mode.

Make the fix for maintainer mode more specific by using
a version dependent pragma in the relevant two C files
only switching off error status for comment warnings.

Can we read this to say the comment error wasn't thrown by GCC 4.1 and 
earlier?


I don't know. The flag was introduced in r1850745 plus r1855446 and 
backported by r1856931. Jim's original commit log only mentions clang.


See PR63633 for more details about the problem caused by that change.


Can we please revert, and add an autoconf test for the support of (or lack
of errors against) this -Wno-error=comment flag? That should allow us to
apply it to all friendly compilers and avoid adding it to other elder 
compilers.


I remember I tried that first and I am not sure what where the exact 
problems I encountered, but I think we didn't have the configure/make 
infrastructure in place to dynamically add CFLAGS for specific modules, 
only LD flags.


Regards,

Rainer

On Thu, Sep 19, 2019 at 10:26 AM Jim Jagielski > wrote:


This breaks building on macOS:

Making all in filters
/Users/jim/src/asf/code/dev/httpd-trunk/srclib/apr/libtool --silent
--mode=link gcc -I/usr/local/include/libxml2 -I/usr/local/include
-Wall -Wmissing-prototypes -Wstrict-prototypes
-Wmissing-declarations -std=c89 -Werror -Wall -Wstrict-prototypes
-Wmissing-prototypes -Wmissing-declarations
-Wdeclaration-after-statement -Wpointer-arith -Wformat
-Wformat-security -Wunused -g -O0 -static      -o libmod_data.la
 mod_data.lo
/Users/jim/src/asf/code/dev/httpd-trunk/srclib/apr/libtool --silent
--mode=link gcc -I/usr/local/include/libxml2 -I/usr/local/include
-Wall -Wmissing-prototypes -Wstrict-prototypes
-Wmissing-declarations -std=c89 -Werror -Wall -Wstrict-prototypes
-Wmissing-prototypes -Wmissing-declarations
-Wdeclaration-after-statement -Wpointer-arith -Wformat
-Wformat-security -Wunused -g -O0 -static      -o
libmod_ratelimit.la  mod_ratelimit.lo
/Users/jim/src/asf/code/dev/httpd-trunk/srclib/apr/libtool --silent
--mode=link gcc -I/usr/local/include/libxml2 -I/usr/local/include
-Wall -Wmissing-prototypes -Wstrict-prototypes
-Wmissing-declarations -std=c89 -Werror -Wall -Wstrict-prototypes
-Wmissing-prototypes -Wmissing-declarations
-Wdeclaration-after-statement -Wpointer-arith -Wformat
-Wformat-security -Wunused -g -O0 -static      -o
libmod_reqtimeout.la  mod_reqtimeout.lo
/Users/jim/src/asf/code/dev/httpd-trunk/srclib/apr/libtool --silent
--mode=link gcc -I/usr/local/include/libxml2 -I/usr/local/include
-Wall -Wmissing-prototypes -Wstrict-prototypes
-Wmissing-declarations -std=c89 -Werror -Wall -Wstrict-prototypes
-Wmissing-prototypes -Wmissing-declarations
-Wdeclaration-after-statement -Wpointer-arith -Wformat
-Wformat-security -Wunused -g -O0 -static      -o
libmod_ext_filter.la  mod_ext_filter.lo
/Users/jim/src/asf/code/dev/httpd-trunk/srclib/apr/libtool --silent
--mode=link gcc -I/usr/local/include/libxml2 -I/usr/local/include
-Wall -Wmissing-prototypes -Wstrict-prototypes
-Wmissing-declarations -std=c89 -Werror -Wall -Wstrict-prototypes
-Wmissing-prototypes -Wmissing-declarations
-Wdeclaration-after-statement -Wpointer-arith -Wformat
-Wformat-security -Wunused -g -O0 -static      -o libmod_request.la
 mod_request.lo
/Users/jim/src/asf/code/dev/httpd-trunk/srclib/apr/libtool --silent
--mode=link gcc -I/usr/local/include/libxml2 -I/usr/local/include
-Wall -Wmissing-prototypes -Wstrict-prototypes
-Wmissing-declarations -std=c89 -Werror -Wall -Wstrict-prototypes
-Wmissing-prototypes -Wmissing-declarations
-Wdeclaration-after-statement -Wpointer-arith -Wformat
-Wformat-security -Wunused -g -O0 -static      -o libmod_include.la
 mod_include.lo
/Users/jim/src/asf/code/dev/httpd-trunk/srclib/apr/libtool --silent
--mode=link gcc -I/usr/local/include/libxml2 -I/usr/local/include
-Wall -Wmissing-prototypes -Wstrict-prototypes
-Wmissing-declarations -std=c89 -Werror -Wall -Wstrict-prototypes
-Wmissing-prototypes -Wmissing-declarations
-Wdeclaration-after-statement -Wpointer-arith -Wformat
-Wformat-security 

Re: [VOTE] Release httpd-2.4.41

2019-08-12 Thread Rainer Jung

Am 09.08.2019 um 15:40 schrieb Daniel Ruggeri:

Hi, all;
Please find below the proposed release tarball and signatures:
https://dist.apache.org/repos/dist/dev/httpd/

I would like to call a VOTE over the next few days to release this 
candidate tarball as 2.4.41:

[X] +1: It's not just good, it's good enough!
[ ] +0: Let's have a talk.
[ ] -1: There's trouble in paradise. Here's what's wrong.

The computed digests of the tarball up for vote are:
sha1: b713e835aa7cde823a4b7f8e3463164f3d9fe63e *httpd-2.4.41.tar.gz
sha256: 3c0f9663240beb0f008acf3b4501c4f339d7467ee345a36c86c46b4d6f3a5461 
*httpd-2.4.41.tar.gz

--
Daniel Ruggeri


+1 to release and thanks a bunch for RM!

Summary: all OK except for

- very few shutdown crashes on Solaris (already observed in 2.4.37, but 
then with MPM event when statically linked, now 2 times both with 
prefork and shared linking)


- not tested but still expected to happen: problems with prefork plus 
mod_ext_filter plus LimitRequestBody on Solaris (not a regression), plus 
indicatons that I also saw it on SLES 11 (and then disabled 
mod_ext_filter there to let the tests proceed).


Detailed report:

- Sigs and hashes OK
- contents of tarballs identical
- contents of tag and tarballs identical
  except for expected deltas

Built on

- Solaris 10 Sparc as 32 Bit Binaries
- SLES 11+12 (64 Bits)
- RHEL 6+7 (64 Bits)

For all platforms built

- with default (shared) and static modules
- with module set reallyall
- using --enable-load-all-modules
- against external APR/APU 1.7.0/1.6.1
  plus for SLES 12 and RHEL 7 also against 1.6.5/1.6.1
  plus for SLES 12 and RHEL 7 built but not tested using APR trunk head

- using external libraries
  - expat 2.2.7
  - pcre 8.43
  - lua 5.3.5 (compiled with LUA_COMPAT_MODULE)
  - distcache 1.5.1
  - libxml2 2.9.9
  - libnghttp2 1.39.1
  - brotli 1.0.7
  - curl 7.65.3
  - jansson 2.12
and
  - openssl 0.9.8zh, 1.0.2, 1.0.2s, 1.0.1e, 1.0.1k, 1.1.1, 1.1.1c plus 
patches (head of a few days ago)


- Tool chain:
- platform gcc except on Solaris
  (gcc 8.3.0 Solaris 10)
- CFLAGS: -O2 -g -Wall -fno-strict-aliasing
  - on Solaris additionally -mpcu=v9, -D_XOPEN_SOURCE,
-D_XOPEN_SOURCE_EXTENDED=1, -D__EXTENSIONS__
and -D_XPG6

All of the 246 builds succeeded.

- compiler warnings: none

Tested for

- Solaris 10, SLES 11+12, RHEL 6+7
- MPMs prefork, worker, event
- default and static module builds
- log level trace8
- module set reallyall
  - for "reallyall" 129 modules plus MPMs
- Perl client bundle build against OpenSSL 1.1.1, 1.1.0i, 1.0.2p and 0.9.8zh
- OpenSSL once linked statically and once as a shared library

Every OpenSSL version in the client tested with every version in the 
server, not just the same version.


The total number of test suite runs was 1926 (plus some on Solaris and 
RHEL 6 still running, the whole suite hasn't finished yet, but enough to 
come up with a clear result).


The following test failures were seen:

a Crashes only on Solaris, this time only with prefork MPM and
  dynamically linked builds.
  The crash seems to happen only at the end of a process during pchild
  clean up and it might be problematic, that the watchdog thread at that
  time still exists.
  gdb info see at end.

b Tests 4, 8 and 12 of t/modules/buffer.t
  Not a regression
  Tests 4, 8 and sometimes 12, always line 37
  Relatively frequent (645) failures on platforms Solaris 10
  and old SLES11, RHEL 6, but not on more modern (and here faster)
  SLES 12 and RHEL 7. Happens for all OpenSSL client and server
  versions and all link types.

c Various tests in t/apache/expr_string.t
  Not a regression.
  18 times,
  Test numbers : 11, 14, 20, 23, 26, 29
  Happens for 18 out of 1926 runs
  (3 times on Solaris 10,
   otherwise always on RHEL6).
  The failure is once on line 101, all others are on line 87.
  They happen, when the error_log contents are checked.
  Could be due to logs written to NFS.

d Test 5 in t/modules/dav.t line 69:
  Not a regression.
  5 times: once RHEL 6, twice RHEL 7 and twice SLES 12
  Creation, modified and now times not in the correct order.
  This seems to be a system issue, all tests done on NFS,
  many tested on virtualized guests.

e Test 54 in t/modules/cgi.t line 232:
  Not a regression
  3 times once Solaris
  Test checks log contents. Could be false positive due to
  logs written to NFS.

Regards,

Rainer

GDB info (sporadic) Solaris shutdown crahes:

#0  0xff07f17c in allocator_free (node=0x1bd8, allocator=0x333530) at 
/path/to/sources/apr/1.7.x/1.7.0/apr-1.7.0/memory/unix/apr_pools.c:486

freelist = 0x0
max_index = 1574824
max_free_index = 512
next = 0x1bd8
index = 5201872
current_free_index = 4284842517
#1  apr_pool_destroy (pool=0x338068) at 
/path/to/sources/apr/1.7.x/1.7.0/apr-1.7.0/memory/unix/apr_pools.c:1043

active = 
allocator = 0x333530
#2  0xfed629d0 in clean_child_exit (code=7) at 

Re: svn commit: r1864464 - in /httpd/httpd/trunk/modules/filters: mod_proxy_html.c mod_xml2enc.c

2019-08-06 Thread Rainer Jung
Thank you. It's funny, that the warning happens directly before the new 
pragma use to influence -Wcomment ...


Regards,

Rainer

Am 06.08.2019 um 09:54 schrieb jor...@apache.org:

Author: jorton
Date: Tue Aug  6 07:54:24 2019
New Revision: 1864464

URL: http://svn.apache.org/viewvc?rev=1864464=rev
Log:
* modules/filters/mod_proxy_html.c, modules/filters/mod_xml2enc.c:
   Fix gcc 9 warnings in code attempting to reduce gcc warnings.
   (should have used expat...)

mod_xml2enc.c:26:28: warning: "/*" within comment [-Wcomment]
26 | /* libxml2 includes unicode/*.h files which uses C++ comments */
   |
mod_proxy_html.c:32:28: warning: "/*" within comment [-Wcomment]
32 | /* libxml2 includes unicode/*.h files which uses C++ comments */
   |

Modified:
 httpd/httpd/trunk/modules/filters/mod_proxy_html.c
 httpd/httpd/trunk/modules/filters/mod_xml2enc.c

Modified: httpd/httpd/trunk/modules/filters/mod_proxy_html.c
URL: 
http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/filters/mod_proxy_html.c?rev=1864464=1864463=1864464=diff
==
--- httpd/httpd/trunk/modules/filters/mod_proxy_html.c (original)
+++ httpd/httpd/trunk/modules/filters/mod_proxy_html.c Tue Aug  6 07:54:24 2019
@@ -29,7 +29,7 @@
  #define VERBOSEB(x) if (verbose) {x}
  #endif
  
-/* libxml2 includes unicode/*.h files which uses C++ comments */

+/* libxml2 includes unicode/[...].h files which uses C++ comments */
  #if defined(__GNUC__)
  #if __GNUC__ > 4 || (__GNUC__ == 4 && __GNUC_MINOR__ >= 6)
  #pragma GCC diagnostic push

Modified: httpd/httpd/trunk/modules/filters/mod_xml2enc.c
URL: 
http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/filters/mod_xml2enc.c?rev=1864464=1864463=1864464=diff
==
--- httpd/httpd/trunk/modules/filters/mod_xml2enc.c (original)
+++ httpd/httpd/trunk/modules/filters/mod_xml2enc.c Tue Aug  6 07:54:24 2019
@@ -23,7 +23,7 @@
  
  #include 
  
-/* libxml2 includes unicode/*.h files which uses C++ comments */

+/* libxml2 includes unicode/[...].h files which uses C++ comments */
  #if defined(__GNUC__)
  #if __GNUC__ > 4 || (__GNUC__ == 4 && __GNUC_MINOR__ >= 6)
  #pragma GCC diagnostic push


Re: svn commit: r1856807 - /httpd/test/framework/trunk/t/security/CVE-2019-0215.t

2019-08-06 Thread Rainer Jung

Am 04.08.2019 um 23:14 schrieb Daniel Ruggeri:


On 8/4/2019 3:30 AM, Rainer Jung wrote:

Hi there,

this one fails for me when the server uses OpenSSL 1.1.1 (no other
variant tested yet) but the client uses something before 1.1.1. In
this case I get Status 500 instead of the expected 403 in the client.

Another older test t/security/CVE-2005-2700.t uses

ok !t_cmp($r->code, 200, "...

instead of

ok t_cmp($r->code, 403, "...

used in the new test. Do others observe the same problem? Should we
relax the condition on 403 or 500, or is it necessary to only relax if
client isn't using 1.1.1 (or maybe depending on effective TLS version)?


I also see the same problem. The 500 must be coming from the LWP client
rather than httpd, though, as httpd does log the 403. I would prefer to
skip the test for non-compatible clients rather than for the internal
client error to be treated as a "pass" of a test it cannot run.


As an intermediate solution I added a request to check, whether TLS 1.3 
works and depending on that switch the expectation to status 403 or 500. 
See r1864463.


I am undecided, whether skipping or allowing 500 is better for the non 
TLS 1.3 case. More opinion? Joe (original author)?


Regards,

Rainer


Re: apr_json.h not found!

2019-08-05 Thread Rainer Jung
As far as I can see it is only part of APR-UTIL 1.7.x, which did not yet 
see any release. You would need to check it out from svn, run buildconf 
in the checkout and then do the usual configure, make, ...


But note it is not released software. Good enough for testing and 
probably a first APR-UTIL 1.7 release is not too far away.


Regards,

Rainer

Am 05.08.2019 um 13:56 schrieb Bill Moo:

Having realised I need to use the buckets and brigades to save the
JSON I am now trying to use the JSON Encode / Decode library but when
I include the apr_json.h file I’m told that it can’t be found and sure
enough a search of the file system proves that the file doesn’t exist
on the file system.

I’m using Ubuntu 18.04 with apache2, -bin, -data, -dev and -utils installed.

If someone can fill in the blanks here I’d be grateful.


Re: svn commit: r1864425 - in /httpd/httpd/trunk/modules/md:md_acme_acct.c md_acme_order.c md_crypt.c md_time.c md_version.h mod_md.cmod_md_config.c mod_md_drive.c

2019-08-05 Thread Rainer Jung

Hi Steffen,

it would help, it you could copy in the warnings here. Not everyone uses 
a compiler setup that shows these warnings. Being explicit about 
observations lowers the bar for us to fix them.


Thanks and regards,

Rainer

Am 05.08.2019 um 12:54 schrieb Steffen:



Also APLOGNO warnings in:

mod_proxy mod_http2 mod_ssl


On Monday 05/08/2019 at 12:32, Rainer Jung  wrote:

Hi Stefan,

Am 05.08.2019 um 12:27 schrieb ic...@apache.org:


Author: icing
Date: Mon Aug  5 10:27:34 2019
New Revision: 1864425
URL: http://svn.apache.org/viewvc?rev=1864425=rev
Log:
 * mod_md: fix compiler warnings


thanks for that. Some trailing spaces have now slipped in though 
(judged on the diff).


Did you notice this report by Gregg:

mod_md.c(386): warning C4003: not enough actual parameters for macro 
'APLOGNO'
mod_md.c(391): warning C4003: not enough actual parameters for macro 
'APLOGNO'
mod_md.c(601): warning C4003: not enough actual parameters for macro 
'APLOGNO'
mod_md.c(608): warning C4003: not enough actual parameters for macro 
'APLOGNO'
mod_md.c(659): warning C4003: not enough actual parameters for macro 
'APLOGNO'
mod_md.c(702): warning C4003: not enough actual parameters for macro 
'APLOGNO'
mod_md.c(912): warning C4003: not enough actual parameters for macro 
'APLOGNO'


Thanks and regards,

Rainer


Re: svn commit: r1864425 - in /httpd/httpd/trunk/modules/md: md_acme_acct.c md_acme_order.c md_crypt.c md_time.c md_version.h mod_md.c mod_md_config.c mod_md_drive.c

2019-08-05 Thread Rainer Jung

Hi Stefan,

Am 05.08.2019 um 12:27 schrieb ic...@apache.org:

Author: icing
Date: Mon Aug  5 10:27:34 2019
New Revision: 1864425

URL: http://svn.apache.org/viewvc?rev=1864425=rev
Log:
  * mod_md: fix compiler warnings


thanks for that. Some trailing spaces have now slipped in though (judged 
on the diff).


Did you notice this report by Gregg:

mod_md.c(386): warning C4003: not enough actual parameters for macro 
'APLOGNO'
mod_md.c(391): warning C4003: not enough actual parameters for macro 
'APLOGNO'
mod_md.c(601): warning C4003: not enough actual parameters for macro 
'APLOGNO'
mod_md.c(608): warning C4003: not enough actual parameters for macro 
'APLOGNO'
mod_md.c(659): warning C4003: not enough actual parameters for macro 
'APLOGNO'
mod_md.c(702): warning C4003: not enough actual parameters for macro 
'APLOGNO'
mod_md.c(912): warning C4003: not enough actual parameters for macro 
'APLOGNO'


Thanks and regards,

Rainer


Compilation warnings in 2.4.40 mod_md

2019-08-05 Thread Rainer Jung
Nothing critcal, just as an info that we should silence them before the 
next release:


/path/to/modules/md/md_time.c:222: warning: ‘percent’ may be used 
uninitialized in this function
/path/to/modules/md/md_time.c:238:65: warning: 'percent' may be used 
uninitialized in this function [-Wmaybe-uninitialized]


and

/path/to/modules/md/mod_md_drive.c:152: warning: ‘rv’ may be used 
uninitialized in this function


I think at least the "percent" ones are false positives, but since we 
are warning free apart from the ones above, it would be nice to silence 
them.


Regards,

Rainer


Re: svn commit: r1856807 - /httpd/test/framework/trunk/t/security/CVE-2019-0215.t

2019-08-04 Thread Rainer Jung

Hi there,

this one fails for me when the server uses OpenSSL 1.1.1 (no other 
variant tested yet) but the client uses something before 1.1.1. In this 
case I get Status 500 instead of the expected 403 in the client.


Another older test t/security/CVE-2005-2700.t uses

ok !t_cmp($r->code, 200, "...

instead of

ok t_cmp($r->code, 403, "...

used in the new test. Do others observe the same problem? Should we 
relax the condition on 403 or 500, or is it necessary to only relax if 
client isn't using 1.1.1 (or maybe depending on effective TLS version)?


Regards,

Rainer

Am 02.04.2019 um 12:44 schrieb jor...@apache.org:

Author: jorton
Date: Tue Apr  2 10:44:12 2019
New Revision: 1856807

URL: http://svn.apache.org/viewvc?rev=1856807=rev
Log:
Add test case for CVE-2019-0215.

Added:
 httpd/test/framework/trunk/t/security/CVE-2019-0215.t

Added: httpd/test/framework/trunk/t/security/CVE-2019-0215.t
URL: 
http://svn.apache.org/viewvc/httpd/test/framework/trunk/t/security/CVE-2019-0215.t?rev=1856807=auto
==
--- httpd/test/framework/trunk/t/security/CVE-2019-0215.t (added)
+++ httpd/test/framework/trunk/t/security/CVE-2019-0215.t Tue Apr  2 10:44:12 
2019
@@ -0,0 +1,26 @@
+use strict;
+use warnings FATAL => 'all';
+
+use Apache::Test;
+use Apache::TestUtil;
+use Apache::TestRequest;
+
+my $vars = Apache::Test::vars();
+
+plan tests => 2, need $vars->{ssl_module_name}, need_lwp,
+qw(LWP::Protocol::https);
+
+Apache::TestRequest::user_agent_keepalive(1);
+Apache::TestRequest::scheme('https');
+Apache::TestRequest::module('ssl_optional_cc');
+
+my $r;
+
+$r = GET "/require/any/";
+
+ok t_cmp($r->code, 403, "first access denied without ccert");
+
+$r = GET "/require/any/";
+
+ok t_cmp($r->code, 403, "second access denied without ccert");
+


Vote thread for 2.4.40 not started yet?

2019-08-03 Thread Rainer Jung

Hi Daniel,

did you forget to start the vote thread or are the uploads not ready yet?

Thanks and regards,

Rainer

Am 02.08.2019 um 22:18 schrieb drugg...@apache.org:

Author: druggeri
Date: Fri Aug  2 20:18:19 2019
New Revision: 35120

Log:
Add 2.4.40 files

Added:
 dev/httpd/CHANGES_2.4
 dev/httpd/CHANGES_2.4.40
 dev/httpd/httpd-2.4.40-deps.tar.bz2   (with props)
 dev/httpd/httpd-2.4.40-deps.tar.bz2.asc
 dev/httpd/httpd-2.4.40-deps.tar.bz2.md5
 dev/httpd/httpd-2.4.40-deps.tar.bz2.sha1
 dev/httpd/httpd-2.4.40-deps.tar.bz2.sha256
 dev/httpd/httpd-2.4.40-deps.tar.gz   (with props)
 dev/httpd/httpd-2.4.40-deps.tar.gz.asc
 dev/httpd/httpd-2.4.40-deps.tar.gz.md5
 dev/httpd/httpd-2.4.40-deps.tar.gz.sha1
 dev/httpd/httpd-2.4.40-deps.tar.gz.sha256
 dev/httpd/httpd-2.4.40.tar.bz2   (with props)
 dev/httpd/httpd-2.4.40.tar.bz2.asc
 dev/httpd/httpd-2.4.40.tar.bz2.md5
 dev/httpd/httpd-2.4.40.tar.bz2.sha1
 dev/httpd/httpd-2.4.40.tar.bz2.sha256
 dev/httpd/httpd-2.4.40.tar.gz   (with props)
 dev/httpd/httpd-2.4.40.tar.gz.asc
 dev/httpd/httpd-2.4.40.tar.gz.md5
 dev/httpd/httpd-2.4.40.tar.gz.sha1
 dev/httpd/httpd-2.4.40.tar.gz.sha256
Modified:
 dev/httpd/Announcement2.4.html
 dev/httpd/Announcement2.4.txt

Modified: dev/httpd/Announcement2.4.html
==
--- dev/httpd/Announcement2.4.html (original)
+++ dev/httpd/Announcement2.4.html Fri Aug  2 20:18:19 2019
@@ -49,7 +49,7 @@
  
  
  

-   Apache HTTP Server 2.4.39 Released
+   Apache HTTP Server 2.4.40 Released
  
  
 September 21, 2018
@@ -57,7 +57,7 @@
  
 The Apache Software Foundation and the Apache HTTP Server Project are
 pleased to https://www.apache.org/dist/httpd/Announcement2.4.html;>announce
-   the release of version 2.4.39 of the Apache
+   the release of version 2.4.40 of the Apache
 HTTP Server ("Apache").  This version of Apache is our latest GA
 release of the new generation 2.4.x branch of Apache HTTPD and
 represents fifteen years of innovation by the project, and is
@@ -69,7 +69,7 @@
 encourage users of all prior versions to upgrade.
  
  
-   Apache HTTP Server 2.4.39 is available for download from:
+   Apache HTTP Server 2.4.40 is available for download from:
  
  
https://httpd.apache.org/download.cgi;
@@ -77,7 +77,7 @@
  
  
 Please see the CHANGES_2.4 file, linked from 
the download page, for a
-   full list of changes.  A condensed list, CHANGES_2.4.39 includes only
+   full list of changes.  A condensed list, CHANGES_2.4.40 includes only
 those changes introduced since the prior 2.4 release.  A summary of all
 of the security vulnerabilities addressed in this and earlier releases
 is available:

Modified: dev/httpd/Announcement2.4.txt
==
--- dev/httpd/Announcement2.4.txt (original)
+++ dev/httpd/Announcement2.4.txt Fri Aug  2 20:18:19 2019
@@ -1,9 +1,9 @@
-Apache HTTP Server 2.4.39 Released
+Apache HTTP Server 2.4.40 Released
  
 September 21, 2018
  
 The Apache Software Foundation and the Apache HTTP Server Project

-   are pleased to announce the release of version 2.4.39 of the Apache
+   are pleased to announce the release of version 2.4.40 of the Apache
 HTTP Server ("Apache").  This version of Apache is our latest GA
 release of the new generation 2.4.x branch of Apache HTTPD and
 represents fifteen years of innovation by the project, and is
@@ -13,7 +13,7 @@
 We consider this release to be the best version of Apache available, and
 encourage users of all prior versions to upgrade.
  
-   Apache HTTP Server 2.4.39 is available for download from:

+   Apache HTTP Server 2.4.40 is available for download from:
  
   https://httpd.apache.org/download.cgi
  
@@ -24,7 +24,7 @@

   https://httpd.apache.org/docs/trunk/new_features_2_4.html
  
 Please see the CHANGES_2.4 file, linked from the download page, for a

-   full list of changes. A condensed list, CHANGES_2.4.39 includes only
+   full list of changes. A condensed list, CHANGES_2.4.40 includes only
 those changes introduced since the prior 2.4 release.  A summary of all
 of the security vulnerabilities addressed in this and earlier releases
 is available:


Re: changelog mod_md ssl patch

2019-08-03 Thread Rainer Jung

Hi Steffen,

Am 03.08.2019 um 12:36 schrieb Steffen:



Changelog says mod_ssl needs patch.

That is a typo or where is the patch.


  *) mod_md: new features
     - supports the ACMEv2 protocol
     - new challenge method 'tls-alpn-01' implemented, needs mod_ssl 
patch to become available


I would say it's conatined in:

  *) mod_ssl/mod_md: reversing dependency by letting mod_ssl offer 
hooks for
 adding certificates and keys to a virtual host. An additional hook 
allows

 answering special TLS connections as used in ACME challenges.
 Adding 2 new hooks for init/get of OCSP stapling status 
information when
 other modules want to provide those. Falls back to own 
implementation with

 same behaviour as before.
 [Stefan Eissing]

especially in "An additional hook allows answering special TLS 
connections as used in ACME challenges.".


The refence to a needed mod_ssl patch is a bit hard tu understand here 
and probably had a historical reason, before that patch actually got 
applied to mod_ssl (and if you are using mod_md from github and mod_ssl 
is older).


Regards,

Rainer


Re: release?

2019-07-20 Thread Rainer Jung

m 20.07.2019 um 10:38 schrieb Marion & Christophe JAILLET:

Hi,

PR60757 and corresponding r1853560 could be a good candidate for backport.

I don't have a configuration for testing so I won't propose it myself 
for backport, but the patch looks simple.


I have added this one (mod_proxy_hcheck in BalancerMember) and two other 
ones ("Mute frequent debug message in mod_proxy_hcheck" and "bytraffic 
needs byrequests") to STATUS right now.


Regards,

Rainer


Le 18/07/2019 à 16:06, Stefan Eissing a écrit :
It would be great if we could make a release this month. There are 
several fixes and improvements already backported and a few 
outstanding issues that need a vote or two.


Please have a look if you find the time. I think Daniel expressed 
willingness to RM this? That'd be great!


Cheers, Stefan


Re: crash during httpd cleanup when using APR debug library (APR_POOL_DEBUG)

2019-07-17 Thread Rainer Jung
Thanks Rüdiger. I hadn't expected it to be fixed in trunk long ago. I 
see now, that there are more useful pool debug backports sitting in 
trunk. Will look at it soon.


Regards,

Rainer

Am 17.07.2019 um 12:09 schrieb Ruediger Pluem:



On 07/17/2019 11:43 AM, Rainer Jung wrote:

Am 17.07.2019 um 10:03 schrieb Ruediger Pluem:



On 07/16/2019 11:28 PM, Rainer Jung wrote:

cross-posted to APR+HTTPD

Crahs happens in #2  0x7faf4c154945 in raise () from /lib64/libc.so.6
#3  0x7faf4c155f21 in abort () from /lib64/libc.so.6
#4  0x7faf4c14d810 in __assert_fail () from /lib64/libc.so.6
#5  0x7faf4c694219 in __pthread_tpp_change_priority () from 
/lib64/libpthread.so.0
#6  0x7faf4c68cd76 in __pthread_mutex_lock_full () from 
/lib64/libpthread.so.0
#7  0x7faf4cd07c29 in apr_thread_mutex_lock (mutex=0x2261fe0) at 
locks/unix/thread_mutex.c:108
#8  0x7faf4cd08603 in apr_pool_walk_tree (pool=0x225a710, fn=0x7faf4cd07fc0 
, data=0x7faf45777c90)
at memory/unix/apr_pools.c:1515
#9  0x7faf4cd08630 in apr_pool_walk_tree (pool=0x6a3ce0, fn=0x7faf4cd07fc0 
, data=0x7faf45777c90) at
memory/unix/apr_pools.c:1521
#10 0x7faf4cd08630 in apr_pool_walk_tree (pool=0x6a3770, fn=0x7faf4cd07fc0 
, data=0x7faf45777c90) at
memory/unix/apr_pools.c:1521
#11 0x7faf4cd08630 in apr_pool_walk_tree (pool=0x6a3110, fn=0x7faf4cd07fc0 
, data=0x7faf45777c90) at
memory/unix/apr_pools.c:1521
#12 0x7faf4cd086df in apr_pool_num_bytes (pool=0x6d81, recurse=) at
memory/unix/apr_pools.c:2304
#13 0x7faf4cd0898f in apr_pool_log_event (pool=0x225a710, event=0x7faf4cd16e74 
"PCALLOC", file_line=0x7faf4cd16d78
"locks/unix/thread_mutex.c:50", deref=-1)
  at memory/unix/apr_pools.c:1543
#14 0x7faf4cd098b8 in apr_pcalloc_debug (pool=0x225a710, size=64, 
file_line=0x7faf4cd16d78
"locks/unix/thread_mutex.c:50") at memory/unix/apr_pools.c:1814
#15 0x7faf4cd07ce5 in apr_thread_mutex_create (mutex=0x225a798, flags=1, 
pool=0x225a710) at
locks/unix/thread_mutex.c:50
#16 0x7faf4cd0a164 in apr_pool_clear_debug (pool=0x225a710, file_line=0x488f09 
"mpm_fdqueue.c:236") at
memory/unix/apr_pools.c:1911
#17 0x0046c455 in ap_queue_info_push_pool (queue_info=0x22648b0, 
pool_to_recycle=0x225a710) at mpm_fdqueue.c:236
#18 0x7faf4bf18821 in process_lingering_close (cs=0x78d670) at event.c:1457
#19 0x7faf4bf196a8 in worker_thread (thd=0x6cae80, dummy=) at event.c:2083
#20 0x7faf4c68b5f0 in start_thread () from /lib64/libpthread.so.0
#21 0x7faf4c1f684d in clone () from /lib64/libc.so.6

So it seems a mutex gets created, which allocates memory, which in turn 
triggers debug logging, which walks pools and
finally tries to lock the not yet initialized lock.

Anyone aware of that? Any ideas how to fix?


This is strange. Before apr_thread_mutex_create is called by apr_pool_clear_debug 
pool->mutex is set to NULL. So IMHO in
frame #7 mutex should be NULL.
Which version of APR are you using?


1.7 with a few debug patches, that should really not make a difference here 
(but might offset line numbers a bit).
1.7.0, 1.7.x, 1.6.5 and 1.6.x do not differ in apr_pools.c.


I was looking at apr trunk. Maybe r1481186 fixes your issue.

Regards

Rüdiger


Re: crash during httpd cleanup when using APR debug library (APR_POOL_DEBUG)

2019-07-17 Thread Rainer Jung

Am 17.07.2019 um 10:03 schrieb Ruediger Pluem:



On 07/16/2019 11:28 PM, Rainer Jung wrote:

cross-posted to APR+HTTPD

Crahs happens in #2  0x7faf4c154945 in raise () from /lib64/libc.so.6
#3  0x7faf4c155f21 in abort () from /lib64/libc.so.6
#4  0x7faf4c14d810 in __assert_fail () from /lib64/libc.so.6
#5  0x7faf4c694219 in __pthread_tpp_change_priority () from 
/lib64/libpthread.so.0
#6  0x7faf4c68cd76 in __pthread_mutex_lock_full () from 
/lib64/libpthread.so.0
#7  0x7faf4cd07c29 in apr_thread_mutex_lock (mutex=0x2261fe0) at 
locks/unix/thread_mutex.c:108
#8  0x7faf4cd08603 in apr_pool_walk_tree (pool=0x225a710, fn=0x7faf4cd07fc0 
, data=0x7faf45777c90)
at memory/unix/apr_pools.c:1515
#9  0x7faf4cd08630 in apr_pool_walk_tree (pool=0x6a3ce0, fn=0x7faf4cd07fc0 
, data=0x7faf45777c90) at
memory/unix/apr_pools.c:1521
#10 0x7faf4cd08630 in apr_pool_walk_tree (pool=0x6a3770, fn=0x7faf4cd07fc0 
, data=0x7faf45777c90) at
memory/unix/apr_pools.c:1521
#11 0x7faf4cd08630 in apr_pool_walk_tree (pool=0x6a3110, fn=0x7faf4cd07fc0 
, data=0x7faf45777c90) at
memory/unix/apr_pools.c:1521
#12 0x7faf4cd086df in apr_pool_num_bytes (pool=0x6d81, recurse=) at memory/unix/apr_pools.c:2304
#13 0x7faf4cd0898f in apr_pool_log_event (pool=0x225a710, event=0x7faf4cd16e74 
"PCALLOC", file_line=0x7faf4cd16d78
"locks/unix/thread_mutex.c:50", deref=-1)
 at memory/unix/apr_pools.c:1543
#14 0x7faf4cd098b8 in apr_pcalloc_debug (pool=0x225a710, size=64, 
file_line=0x7faf4cd16d78
"locks/unix/thread_mutex.c:50") at memory/unix/apr_pools.c:1814
#15 0x7faf4cd07ce5 in apr_thread_mutex_create (mutex=0x225a798, flags=1, 
pool=0x225a710) at
locks/unix/thread_mutex.c:50
#16 0x7faf4cd0a164 in apr_pool_clear_debug (pool=0x225a710, file_line=0x488f09 
"mpm_fdqueue.c:236") at
memory/unix/apr_pools.c:1911
#17 0x0046c455 in ap_queue_info_push_pool (queue_info=0x22648b0, 
pool_to_recycle=0x225a710) at mpm_fdqueue.c:236
#18 0x7faf4bf18821 in process_lingering_close (cs=0x78d670) at event.c:1457
#19 0x7faf4bf196a8 in worker_thread (thd=0x6cae80, dummy=) at event.c:2083
#20 0x7faf4c68b5f0 in start_thread () from /lib64/libpthread.so.0
#21 0x7faf4c1f684d in clone () from /lib64/libc.so.6

So it seems a mutex gets created, which allocates memory, which in turn 
triggers debug logging, which walks pools and
finally tries to lock the not yet initialized lock.

Anyone aware of that? Any ideas how to fix?


This is strange. Before apr_thread_mutex_create is called by apr_pool_clear_debug 
pool->mutex is set to NULL. So IMHO in
frame #7 mutex should be NULL.
Which version of APR are you using?


1.7 with a few debug patches, that should really not make a difference 
here (but might offset line numbers a bit). 1.7.0, 1.7.x, 1.6.5 and 
1.6.x do not differ in apr_pools.c.


The code that jumps into the mutex creation is:

1877 APR_DECLARE(void) apr_pool_clear_debug(apr_pool_t *pool,
1878const char *file_line)
1879 {
...
1902
1903 pool_clear_debug(pool, file_line);
1904
1905 #if APR_HAS_THREADS
1906 /* If we had our own mutex, it will have been destroyed by
1907  * the registered cleanups.  Recreate the mutex.  Unlock
1908  * the mutex we obtained above.
1909  */
1910 if (mutex != pool->mutex) {
1911 (void)apr_thread_mutex_create(>mutex,
1912   APR_THREAD_MUTEX_NESTED, pool);
1913
1914 if (mutex != NULL)
1915 (void)apr_thread_mutex_unlock(mutex);
1916 }
1917 #endif /* APR_HAS_THREADS */
1918 }

and in pool_clear_debug I see overwriting of pool allocated bytes with a 
'A' to poison data but I don't see, where pool->mutex is nulled.


Here's data inspection from gdb for pool, pool->parent (pconf), 
pool->mutex and pool->parent->mutex. IMHO one can see, that pool->mutex 
has indeed been treated by poison 'A'.


(gdb) print *pool
$1 = {parent = 0x6a3ce0, child = 0x0, sibling = 0x7729b0, ref = 
0x6a3ce8, cleanups = 0x0, free_cleanups = 0x0, allocator = 0x784a20, 
subprocesses = 0x0,
  abort_fn = 0x437920 , user_data = 0x0, tag = 
0x7faf4bf1d62a "transaction", joined = 0x0, nodes = 0x9335b0, file_line 
= 0x7faf4bf1d61d "event.c:1829",
  creation_flags = 0, stat_alloc = 1, stat_total_alloc = 45, stat_clear 
= 1, owner = 140390560073488, mutex = 0x2261fe0, pre_cleanups = 0x0}


(gdb) print *pool->parent
$2 = {parent = 0x6a3770, child = 0x225a710, sibling = 0x0, ref = 
0x6c2c70, cleanups = 0x0, free_cleanups = 0x0, allocator = 0x0, 
subprocesses = 0x0,
  abort_fn = 0x437920 , user_data = 0x7911d0, tag = 
0x47cb57 "pconf", joined = 0x0, nodes = 0x227eaf0, file_line = 0x47cb4c 
"main.c:296", creation_flags = 0,
  stat_alloc = 86248, stat_total_alloc = 172527, stat_clear = 1, owner 
= 140390893647616, mutex = 0x6a31b0, pre_cleanups = 0x0}


(gd

crash during httpd cleanup when using APR debug library (APR_POOL_DEBUG)

2019-07-16 Thread Rainer Jung

cross-posted to APR+HTTPD

Crahs happens in #2  0x7faf4c154945 in raise () from /lib64/libc.so.6
#3  0x7faf4c155f21 in abort () from /lib64/libc.so.6
#4  0x7faf4c14d810 in __assert_fail () from /lib64/libc.so.6
#5  0x7faf4c694219 in __pthread_tpp_change_priority () from 
/lib64/libpthread.so.0
#6  0x7faf4c68cd76 in __pthread_mutex_lock_full () from 
/lib64/libpthread.so.0
#7  0x7faf4cd07c29 in apr_thread_mutex_lock (mutex=0x2261fe0) at 
locks/unix/thread_mutex.c:108
#8  0x7faf4cd08603 in apr_pool_walk_tree (pool=0x225a710, 
fn=0x7faf4cd07fc0 , data=0x7faf45777c90) at 
memory/unix/apr_pools.c:1515
#9  0x7faf4cd08630 in apr_pool_walk_tree (pool=0x6a3ce0, 
fn=0x7faf4cd07fc0 , data=0x7faf45777c90) at 
memory/unix/apr_pools.c:1521
#10 0x7faf4cd08630 in apr_pool_walk_tree (pool=0x6a3770, 
fn=0x7faf4cd07fc0 , data=0x7faf45777c90) at 
memory/unix/apr_pools.c:1521
#11 0x7faf4cd08630 in apr_pool_walk_tree (pool=0x6a3110, 
fn=0x7faf4cd07fc0 , data=0x7faf45777c90) at 
memory/unix/apr_pools.c:1521
#12 0x7faf4cd086df in apr_pool_num_bytes (pool=0x6d81, 
recurse=) at memory/unix/apr_pools.c:2304
#13 0x7faf4cd0898f in apr_pool_log_event (pool=0x225a710, 
event=0x7faf4cd16e74 "PCALLOC", file_line=0x7faf4cd16d78 
"locks/unix/thread_mutex.c:50", deref=-1)

at memory/unix/apr_pools.c:1543
#14 0x7faf4cd098b8 in apr_pcalloc_debug (pool=0x225a710, size=64, 
file_line=0x7faf4cd16d78 "locks/unix/thread_mutex.c:50") at 
memory/unix/apr_pools.c:1814
#15 0x7faf4cd07ce5 in apr_thread_mutex_create (mutex=0x225a798, 
flags=1, pool=0x225a710) at locks/unix/thread_mutex.c:50
#16 0x7faf4cd0a164 in apr_pool_clear_debug (pool=0x225a710, 
file_line=0x488f09 "mpm_fdqueue.c:236") at memory/unix/apr_pools.c:1911
#17 0x0046c455 in ap_queue_info_push_pool (queue_info=0x22648b0, 
pool_to_recycle=0x225a710) at mpm_fdqueue.c:236
#18 0x7faf4bf18821 in process_lingering_close (cs=0x78d670) at 
event.c:1457
#19 0x7faf4bf196a8 in worker_thread (thd=0x6cae80, dummy=optimized out>) at event.c:2083

#20 0x7faf4c68b5f0 in start_thread () from /lib64/libpthread.so.0
#21 0x7faf4c1f684d in clone () from /lib64/libc.so.6

So it seems a mutex gets created, which allocates memory, which in turn 
triggers debug logging, which walks pools and finally tries to lock the 
not yet initialized lock.


Anyone aware of that? Any ideas how to fix?

Thanks and regards,

Rainer


Re: svn commit: r1859376 - in /httpd/httpd/branches/2.4.x: ./ CHANGES STATUS modules/filters/mod_reqtimeout.c

2019-05-16 Thread Rainer Jung
This and the previous change I forgot to first move to the accepted 
section in STATUS. They had enough votes (for quite some time), so I 
hope it is OK to shortcut the procedure for now. It was only an oversight.


Thanks (and sorry),

Rainer

Am 16.05.2019 um 15:09 schrieb rj...@apache.org:

Author: rjung
Date: Thu May 16 13:09:35 2019
New Revision: 1859376

URL: http://svn.apache.org/viewvc?rev=1859376=rev
Log:
mod_reqtimeout: Fix default rates missing (not applied) in 2.4.39.
PR 63325.

Backport of r1857129 + r1857130 from trunk.
Proposed by: ylavic
Backported by: rjung
Reviewed by: ylavic, rpluem, jim

Modified:
 httpd/httpd/branches/2.4.x/   (props changed)
 httpd/httpd/branches/2.4.x/CHANGES
 httpd/httpd/branches/2.4.x/STATUS
 httpd/httpd/branches/2.4.x/modules/filters/mod_reqtimeout.c

Propchange: httpd/httpd/branches/2.4.x/
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Thu May 16 13:09:35 2019
@@ -8,4 +8,4 @@
  /httpd/httpd/branches/trunk-md:1804087-1804529
  /httpd/httpd/branches/trunk-override-index:1793921-1793931
  /httpd/httpd/branches/wombat-integration:723609-723841
...

Modified: httpd/httpd/branches/2.4.x/CHANGES
URL: 
http://svn.apache.org/viewvc/httpd/httpd/branches/2.4.x/CHANGES?rev=1859376=1859375=1859376=diff
==
--- httpd/httpd/branches/2.4.x/CHANGES [utf-8] (original)
+++ httpd/httpd/branches/2.4.x/CHANGES [utf-8] Thu May 16 13:09:35 2019
@@ -1,6 +1,9 @@
   -*- coding: utf-8 -*-
  Changes with Apache 2.4.40
  
+  *) mod_reqtimeout: Fix default rates missing (not applied) in 2.4.39.

+ PR 63325. [Yann Ylavic]
+
*) mod_info: Fix output of server settings for PIPE_BUF in mod_info in
   the rare case that PIPE_BUF is defined. [Rainer Jung]
  


Modified: httpd/httpd/branches/2.4.x/STATUS
URL: 
http://svn.apache.org/viewvc/httpd/httpd/branches/2.4.x/STATUS?rev=1859376=1859375=1859376=diff
==
--- httpd/httpd/branches/2.4.x/STATUS (original)
+++ httpd/httpd/branches/2.4.x/STATUS Thu May 16 13:09:35 2019
@@ -215,12 +215,6 @@ PATCHES PROPOSED TO BACKPORT FROM TRUNK:
   2.4.x patch: 
http://people.apache.org/~ylavic/patches/httpd-2.4.x-forward_100_continue-v5.patch
   +1: ylavic, jim
  
-  *) mod_reqtimeout: Fix default rates missing in 2.4.39. PR 63325

- trunk patch: http://svn.apache.org/r1857129
-  http://svn.apache.org/r1857130
- 2.4.x patch: svn merge -c 1857129,1857130 ^/httpd/httpd/trunk .
- +1: ylavic, rpluem, jim
-
*) mod_status: PR60647: ACC per connection not available w/ event MPM
   trunk patch: http://svn.apache.org/r1780280
   2.4.x patch: svn merge -c 1780280 ^/httpd/httpd/trunk .


...


Re: [Bug 60757] mod_proxy_hcheck Doesn't perform checks

2019-03-30 Thread Rainer Jung

Hi Jean-Frederic,

Am 14.02.2019 um 09:15 schrieb root:

https://bz.apache.org/bugzilla/show_bug.cgi?id=60757

jfclere  changed:

What|Removed |Added

  Status|NEEDINFO|RESOLVED
  Resolution|--- |FIXED

--- Comment #20 from jfclere  ---
Fixed by r1853560 in trunk, will propose a back port later.


did you plan to propose that backport? r1853560 and the unrelated 
r1853992 (same file) have not been backported yet.


Regards,

Rainer



Re: [VOTE] Release httpd-2.4.39

2019-03-30 Thread Rainer Jung

Am 27.03.2019 um 16:09 schrieb Daniel Ruggeri:

Hi, all;
    Please find below the proposed release tarball and signatures:
https://dist.apache.org/repos/dist/dev/httpd/

I would like to call a VOTE over the next few days to release this 
candidate tarball as 2.4.39:

[X] +1: It's not just good, it's good enough!
[ ] +0: Let's have a talk.
[ ] -1: There's trouble in paradise. Here's what's wrong.

The computed digests of the tarball up for vote are:
sha1: e66d6bfea42254e64d3b5009f49ecc486ac46de2 *httpd-2.4.39.tar.gz
sha256: 8b95fe249f3a6c50aad3ca125eef3e02d619116cde242e1bc3c266b7b5c37c30 
*httpd-2.4.39.tar.gz


+1 to release and thanks a bunch for RM!

Summary: all OK except for

- some shutdown crashes on Solaris with MPM event when statically linked 
(already observed in 2.4.37)


- not tested but still expected to happen: problems with prefork plus 
mod_ext_filter plus LimitRequestBody  on Solaris (not a regression)


Detailed report:

- Sigs and hashes OK
- contents of tarballs identical
- contents of tag and tarballs identical
  except for expected deltas

Built on

- Solaris 10 Sparc as 32 Bit Binaries
- SLES 11+12 (64 Bits)
- RHEL 6+7 (64 Bits)

For all platforms built

- with default (shared) and static modules
- with module set reallyall
- using --enable-load-all-modules
- against external APR/APU 1.6.5/1.6.1

- using external libraries
  - expat 2.2.6
  - pcre 8.43
  - lua 5.3.5 (compiled with LUA_COMPAT_MODULE)
  - distcache 1.5.1
  - libxml2 2.9.9
  - libnghttp2 1.37.0
  - brotli 1.0.7
  - curl 7.64.1
  - jansson 2.12
and
  - openssl 0.9.8zh, 1.0.2, 1.0.2r plus patches (head), 1.0.1e, 1.0.1j 
plus patches (head), 1.1.1, 1.1.1b plus patches (head)


- Tool chain:
- platform gcc except on Solaris
  (gcc 8.2.0 Solaris 10)
- CFLAGS: -O2 -g -Wall -fno-strict-aliasing
  - on Solaris additionally -mpcu=v9, -D_XOPEN_SOURCE,
-D_XOPEN_SOURCE_EXTENDED=1, -D__EXTENSIONS__
and -D_XPG6

All of the 126 builds succeeded.

- compiler warnings: none

Tested for

- Solaris 10, SLES 11+12, RHEL 6+7
- MPMs prefork, worker, event
- default and static module builds
- log level trace8
- module set reallyall
  - for "reallyall" 129 modules plus MPMs
- Perl client bundle build against OpenSSL 1.1.1, 1.1.0i, 1.0.2p and 0.9.8zh
- OpenSSL once linked statically and once as a shared library

Every OpenSSL version in the client tested with every version in the 
server, not just the same version.


The total number of test suite runs was 1366 (plus some on Solaris still 
running, the whole suite hasn't finished yet, but enough to come up with 
a clear expectation).


The following test failures were seen:

a Crashes only on Solaris and only with event MPM and static builds.
  The crash seems to happen only at the end of a process, likely due
  to double cleanup of the various OpenSSL instances that are
  contained in the process.

b Tests 4, 8 and 12 of t/modules/buffer.t
  Not a regression
  Relatively frequent (725) failures on all platforms for all OpenSSL
  client and server versions.
  See earlier list discussions about buffer.t.

c Various tests in t/apache/expr_string.t
  Not a regression.
  Test numbers : 6, 11, 14, 17, 20, 23, 26, 29
  Happens for 61 out of about 1350 runs
  (4 times on SLES 11, 2 times on Solaris 10,
   otherwise always on RHEL6).
  The failure is always on line 87, where the error_log contents
  are checked. Could be due to logs written to NFS.

d Test 5 in t/modules/dav.t:
  Not a regression.
  once RHEL 6 and once SLES 11
  Creation, modified and now times not in the correct order.
  This seems to be a system issue, all tests done on NFS,
  many tested on virtualized guests.

e Test 54 in t/modules/cgi.t line 232:
  Not a regression
  8 times once Solaris
  Test checks log contents. Could be false positive due to
  logs written to NFS.

Regards,

Rainer


Re: t/ssl/ocsp.t

2019-02-05 Thread Rainer Jung

Am 05.02.2019 um 11:33 schrieb Joe Orton:

On Thu, Jan 17, 2019 at 09:02:02PM +0100, Christophe JAILLET wrote:

Hi,

I see test errors in #1 and #3 in t/ssl/ocsp.t.

Does anyone else see it?


I see it too.  I changed it as you suggested in r1852984, maybe Rainer
will comment if it breaks things for for him again.


No, looks good.

In my environments, content and message are the same, so old and new 
version of the test is successful. The only small delta between content 
and message is when using client plus server with OpenSSL 1.1.1, then 
the message is


  Status read failed:

and content

  Status read failed:  at /path/to/Net/HTTP/Methods.pm line 282.

Which doesn't matter in the regexp those strings are checked against.

I have not updated my bundle installation since mid september 2018 (plus 
patches at that time to make it work with TLS 1.3).


When either client or server are below 1.1.1, messages/content for the 
two test cases is more precise:


Can't connect to localhost:8535 (SSL connect attempt failed because of 
handshake problems error:14094410:SSL routines:ssl3_read_bytes:sslv3 
alert handshake failure)


and

Can't connect to localhost:8535 (SSL connect attempt failed because of 
handshake problems error:14094410:SSL routines:ssl3_read_bytes:sslv3 
alert certificate revoked)


Depending on the OpenSSL version, the error number can also be 14094414 
and the method SSL3_READ_BYTES (upper case). Maybe in a more modern or 
future bundle setup, TLS 1.3 based tests will be back to also showing 
the real reason and not just "read failed".


Anyways, any of those variations match the regexp which is used in the 
test. No problem with the updated test here.



I was seeing this with openssl-1.1.1a and the Fedora IO::Socket::SSL is
at 2.060 but with a bunch of OpenSSL 1.1.1/TLSv1.3 patches applied,
which might well make a difference.

Regards, Joe



Looking deeper at the output (i.e. --verbose), it looks like the issue is in
the test itself.
All conditions seem to be there, but I need to turn:
    my $message = $r->message();
into:
    my $message = $r->content();

in both tests to have them pass.

Is it expected?


I don't remind issues with this test in the past.
This part of the test has been changed in r1844479.


Regards,

Rainer


Solaris, prefork, accept mutex and mod_ext_filter (Was: Prefork MPM and mod_watchdog)

2019-02-03 Thread Rainer Jung

Am 31.01.2019 um 10:31 schrieb Stefan Eissing:




Am 27.01.2019 um 14:40 schrieb Rainer Jung :

- as soon as I enable mod_watchdog only (but not the above modules that would 
use it), the hangs start to happen every now and then.


Hmm, that sounds strange. As I understood the code, none of the parent/child 
watchdogs would be active then. So, its child_init should not do anything 
either.

But, if one of the proxy monitors runs against the server itself, could it 
connect against the child process it (the watchdog) is running on?


mod_watchdog was a red herring.

I can now frequently reproduce running t/modules/ext_filter.t only. I 
stripped the reproducer down to the part of t/modules/ext_filter.t where 
it runs


POST "/apache/extfilter/out-limit/modules/cgi/perl_echo.pl", content => 
"foo and bar\n"


10 times in a row. If I increase the iteration count slightly to 20, the 
problem happens nearly every time. I also replaced perl_echo.pl and 
eval-cmd.pl by small C programs doing the echo and the s/foo/bar/ and 
can still reproduce.


This test incolved mod_ext_filter and LimitRequestBody.

It seems I can not reproduce, if I shorten the POST body, so that it no 
longer hits the LimitRequestBody 6 configured for 
/apache/extfilter/out-limit/.


What happens, but I do not understand is:

- the first few requests are handled by two children in an alternating 
way. I can see the accept mutex being passed between these two children 
using lwp_mutex_timedlock(ADDRESS, 0x) and 
lwp_mutex_unlock(ADDRESS) using truss (Solaris variant of strace). So 
Solaris seems to implement the pthread mutexes via these lwp mutexes.


- after a few requests alternating between the two children, something 
strange happens:


  - child A returns from port_getn() for 22 (probably part of the 
accept() impl)

  - child A calls accept()
  - child A unlocks the accept mutex using lwp_mutex_unlock()
  - child B locks the accept mutex using lwp_mutex_timedlock()
  - child B calls port_associate for 22 (probably part of accept() impl)
  - child A handles the request
! - child A also calls port_associate for 22 (no sign of lock acquire)
! - child A returns from port_getn() for 22
  - child A calls accept() and starts to handle the connection
! - child B also returns from port_getn() for 22
! - child B gets EAGAIN for the accept()
  - child B calls port_associate for 22
  - child A handles the request
! - child A gets EDEADLK for pthread_mutex_lock() (this is from the 
error_log; there's no system call for this instance of 
pthread_mutex_lock() in the truss output). EDEADLK for 
pthread_mutex_lock() means that this process already has that lock.

  - child B returns from port_getn() for 22
  - child B calls accept() and starts to handle the connection
  - child A exits (now B is the only child left)
  - child B proceeds request handling
  - child B does all further port_associate(), port_getn(), and 
accept() plus connection and request handling. No more 
lwp_mutex_timedlock() or lwp_mutex_unlock() for B, maybe due to some 
optimization when only one process for a lock is left.



It is quite possible, that there is an underlying lwp_mutex or portfs 
bug here. But it is strange, that this only comes up when used with 
mod_ext_filter in combination with LimitRequestBody.


There was the fix

https://bz.apache.org/bugzilla/show_bug.cgi?id=60375

but I don't see, how this would influence the above list.

I can try to further narrow down (using static content to eliminate one 
forked child; using LimitRequestBody without mod_exit-filter etc.), but 
maybe someone already has an idea?


Regards,

Rainer


Re: Prefork MPM and mod_watchdog

2019-01-27 Thread Rainer Jung
Only about the part: "the limit of 3 child processes and the 
MaxRequestWorkers warning". That happens also on Linux. The sizing is 
part of the test framework and is insufficient, because apart from one 
child for a request plus another one for a proxied request on the 
backend, more child processes can be busy while sitting in 
KeepAliveTimeout from previous proxied requests (we are using persistent 
backend connections). The timeout is 5 seconds.


I added a little headroom to the prefork sizing in r1852306 to 
https://svn.apache.org/repos/asf/perl/Apache-Test/trunk.


The real problem, pthread mutex failures on Solaris with prefork and 
mod_watchdog is still open.


Regards,

Rainer

Am 27.01.2019 um 14:40 schrieb Rainer Jung:

Hi all,

since around 2.4.26 I notice occasional hangs during release testing 
only on Solaris 10 Sparc when using MPM prefork.


Those hangs were always observed during proxy testing. The situation was 
in most cases (all?), that only one running httpd process was left, so 
it would accept the incoming request but could not proxy them and each 
request ran into the proxy timeout (I think 1 minute in the test). The 
process was in fact an httpd child process. Parent process and other 
children were gone.


I now narrowed it down some more. I am not sure, this is the only 
situation, but at least this is one I do observe most often:


- no problems with event or worker MPM

- no problems on REHL and SLES (Linux)

- even on Solaris, running without mod_watchdog (and therefore without 
mod_heartbeat, mod_heartmonitor, mod_lbmethod_heartbeat and 
mod_proxy_hcheck) it seems the hang does not occur.


- as soon as I enable mod_watchdog only (but not the above modules that 
would use it), the hangs start to happen every now and then.


The pattern seems to be (from the error log):

[Sat Jan 26 21:36:01.855207 2019] [core:trace4] [pid 7850] 
mpm_common.c(536): mpm child 7853 (gen 0/slot 0) started
[Sat Jan 26 21:36:01.875826 2019] [mpm_prefork:notice] [pid 7850] 
AH00163: Apache/2.4.38 (Unix) OpenSSL/1.0.2 configured -- resuming 
normal operations
[Sat Jan 26 21:36:01.875975 2019] [mpm_prefork:info] [pid 7850] AH00164: 
Server built: Jan 18 2019 17:54:50
[Sat Jan 26 21:36:01.876208 2019] [core:notice] [pid 7850] AH00094: 
Command line: '/path/to/bin/httpd -d /path/to/t -f 
/path/to/t/conf/httpd.conf -D APACHE2 -D APACHE2_4'
[Sat Jan 26 21:36:01.876340 2019] [core:debug] [pid 7850] log.c(1571): 
AH02639: Using SO_REUSEPORT: no (1)
[Sat Jan 26 21:36:01.876417 2019] [mpm_prefork:debug] [pid 7850] 
prefork.c(923): AH00165: Accept mutex: pthread (default: pthread)
[Sat Jan 26 21:36:01.875753 2019] [core:trace4] [pid 7850] 
mpm_common.c(536): mpm child 7854 (gen 0/slot 1) started

...
[Sat Jan 26 21:36:18.969742 2019] [core:trace4] [pid 7850] 
mpm_common.c(536): mpm child 7889 (gen 0/slot 2) started

...
SHOULD BE FINE UNTIL HERE
...
[Sat Jan 26 21:38:29.016961 2019] [mpm_prefork:error] [pid 7850] 
AH00161: server reached MaxRequestWorkers setting, consider raising the 
MaxRequestWorkers setting

...
UNCLEAR, WHETHER THAT IS UNEXPECTED, BUT THEN THE REAL ERROR
...
[Sat Jan 26 21:42:48.213005 2019] [mpm_prefork:emerg] [pid 7889] 
(45)Deadlock situation detected/avoided: AH00144: couldn't grab the 
accept mutex
[Sat Jan 26 21:42:48.490884 2019] [mpm_prefork:emerg] [pid 7853] 
(45)Deadlock situation detected/avoided: AH00144: couldn't grab the 
accept mutex

...
[Sat Jan 26 21:42:49.129401 2019] [core:alert] [pid 7850] AH00050: Child 
7853 returned a Fatal error... Apache is exiting!
[Sat Jan 26 21:42:49.129519 2019] [:emerg] [pid 7850] AH02818: MPM run 
failed, exiting


bin/httpd -V
Server version: Apache/2.4.38 (Unix)
Server built:   Jan 18 2019 17:54:50
Server's Module Magic Number: 20120211:83
Server loaded:  APR 1.6.5, APR-UTIL 1.6.1
Compiled using: APR 1.6.5, APR-UTIL 1.6.1
Architecture:   32-bit
Server MPM: prefork
   threaded: no
     forked: yes (variable process count)
Server compiled with
  -D APR_HAS_SENDFILE
  -D APR_HAS_MMAP
  -D APR_HAVE_IPV6 (IPv4-mapped addresses enabled)
  -D APR_USE_PROC_PTHREAD_SERIALIZE
  -D APR_USE_PTHREAD_SERIALIZE
  -D SINGLE_LISTEN_UNSERIALIZED_ACCEPT
  -D APR_HAS_OTHER_CHILD
  -D AP_HAVE_RELIABLE_PIPED_LOGS
  -D DYNAMIC_MODULE_LIMIT=256
  -D HTTPD_ROOT="..."
  -D SUEXEC_BIN=".../bin/suexec"
  -D DEFAULT_PIDLOG="logs/httpd.pid"
  -D DEFAULT_SCOREBOARD="logs/apache_runtime_status"
  -D DEFAULT_ERRORLOG="logs/error_log"
  -D AP_TYPES_CONFIG_FILE="conf/mime.types"
  -D SERVER_CONFIG_FILE="conf/httpd.conf"

I remember Solaris having a very bad deadlock detection but that goes 
back to using fcntl locks. Here we are using pthread locks. Since this 
is APR 1.6, my old observation about pthread timed lock problems on 
Solaris 10 also does not apply here.


I wonder what is special about prefork and mod_watchdog.

In addition, what I find strange is, that the acce

Prefork MPM and mod_watchdog

2019-01-27 Thread Rainer Jung

Hi all,

since around 2.4.26 I notice occasional hangs during release testing 
only on Solaris 10 Sparc when using MPM prefork.


Those hangs were always observed during proxy testing. The situation was 
in most cases (all?), that only one running httpd process was left, so 
it would accept the incoming request but could not proxy them and each 
request ran into the proxy timeout (I think 1 minute in the test). The 
process was in fact an httpd child process. Parent process and other 
children were gone.


I now narrowed it down some more. I am not sure, this is the only 
situation, but at least this is one I do observe most often:


- no problems with event or worker MPM

- no problems on REHL and SLES (Linux)

- even on Solaris, running without mod_watchdog (and therefore without 
mod_heartbeat, mod_heartmonitor, mod_lbmethod_heartbeat and 
mod_proxy_hcheck) it seems the hang does not occur.


- as soon as I enable mod_watchdog only (but not the above modules that 
would use it), the hangs start to happen every now and then.


The pattern seems to be (from the error log):

[Sat Jan 26 21:36:01.855207 2019] [core:trace4] [pid 7850] 
mpm_common.c(536): mpm child 7853 (gen 0/slot 0) started
[Sat Jan 26 21:36:01.875826 2019] [mpm_prefork:notice] [pid 7850] 
AH00163: Apache/2.4.38 (Unix) OpenSSL/1.0.2 configured -- resuming 
normal operations
[Sat Jan 26 21:36:01.875975 2019] [mpm_prefork:info] [pid 7850] AH00164: 
Server built: Jan 18 2019 17:54:50
[Sat Jan 26 21:36:01.876208 2019] [core:notice] [pid 7850] AH00094: 
Command line: '/path/to/bin/httpd -d /path/to/t -f 
/path/to/t/conf/httpd.conf -D APACHE2 -D APACHE2_4'
[Sat Jan 26 21:36:01.876340 2019] [core:debug] [pid 7850] log.c(1571): 
AH02639: Using SO_REUSEPORT: no (1)
[Sat Jan 26 21:36:01.876417 2019] [mpm_prefork:debug] [pid 7850] 
prefork.c(923): AH00165: Accept mutex: pthread (default: pthread)
[Sat Jan 26 21:36:01.875753 2019] [core:trace4] [pid 7850] 
mpm_common.c(536): mpm child 7854 (gen 0/slot 1) started

...
[Sat Jan 26 21:36:18.969742 2019] [core:trace4] [pid 7850] 
mpm_common.c(536): mpm child 7889 (gen 0/slot 2) started

...
SHOULD BE FINE UNTIL HERE
...
[Sat Jan 26 21:38:29.016961 2019] [mpm_prefork:error] [pid 7850] 
AH00161: server reached MaxRequestWorkers setting, consider raising the 
MaxRequestWorkers setting

...
UNCLEAR, WHETHER THAT IS UNEXPECTED, BUT THEN THE REAL ERROR
...
[Sat Jan 26 21:42:48.213005 2019] [mpm_prefork:emerg] [pid 7889] 
(45)Deadlock situation detected/avoided: AH00144: couldn't grab the 
accept mutex
[Sat Jan 26 21:42:48.490884 2019] [mpm_prefork:emerg] [pid 7853] 
(45)Deadlock situation detected/avoided: AH00144: couldn't grab the 
accept mutex

...
[Sat Jan 26 21:42:49.129401 2019] [core:alert] [pid 7850] AH00050: Child 
7853 returned a Fatal error... Apache is exiting!
[Sat Jan 26 21:42:49.129519 2019] [:emerg] [pid 7850] AH02818: MPM run 
failed, exiting


bin/httpd -V
Server version: Apache/2.4.38 (Unix)
Server built:   Jan 18 2019 17:54:50
Server's Module Magic Number: 20120211:83
Server loaded:  APR 1.6.5, APR-UTIL 1.6.1
Compiled using: APR 1.6.5, APR-UTIL 1.6.1
Architecture:   32-bit
Server MPM: prefork
  threaded: no
forked: yes (variable process count)
Server compiled with
 -D APR_HAS_SENDFILE
 -D APR_HAS_MMAP
 -D APR_HAVE_IPV6 (IPv4-mapped addresses enabled)
 -D APR_USE_PROC_PTHREAD_SERIALIZE
 -D APR_USE_PTHREAD_SERIALIZE
 -D SINGLE_LISTEN_UNSERIALIZED_ACCEPT
 -D APR_HAS_OTHER_CHILD
 -D AP_HAVE_RELIABLE_PIPED_LOGS
 -D DYNAMIC_MODULE_LIMIT=256
 -D HTTPD_ROOT="..."
 -D SUEXEC_BIN=".../bin/suexec"
 -D DEFAULT_PIDLOG="logs/httpd.pid"
 -D DEFAULT_SCOREBOARD="logs/apache_runtime_status"
 -D DEFAULT_ERRORLOG="logs/error_log"
 -D AP_TYPES_CONFIG_FILE="conf/mime.types"
 -D SERVER_CONFIG_FILE="conf/httpd.conf"

I remember Solaris having a very bad deadlock detection but that goes 
back to using fcntl locks. Here we are using pthread locks. Since this 
is APR 1.6, my old observation about pthread timed lock problems on 
Solaris 10 also does not apply here.


I wonder what is special about prefork and mod_watchdog.

In addition, what I find strange is, that the accept mutex error lead to 
child crashes, but not to a start of new child processes. Instead one of 
the two child crashes (the second one) seems to be noticed by the parent:


[core:alert] [pid 7850] AH00050: Child 7853 returned a Fatal error... 
Apache is exiting!

[:emerg] [pid 7850] AH02818: MPM run failed, exiting

and in fact the parent goes away but the third and last child process 
stays and continues to handle requests. That doesn't seem right.


I also see complains about reaching MaxRequestWorkers in the prefork 
tests on Solaris. I haven't verified it, but it seems they run with a 
fixed limit of three child processes. On the one hand, I would only 
expect two of them to be actually used at any time (request plus proxied 
request), since the tests are not executed in parallel. So the message 
is a bit strange. On the 

Re: svn commit: r1851794 [1/37] - in /httpd/httpd/trunk/docs/manual: ./ developer/ howto/ misc/ mod/ platform/ programs/ rewrite/ ssl/ vhosts/

2019-01-22 Thread Rainer Jung

Am 22.01.2019 um 15:27 schrieb Eric Covener:

Modified: httpd/httpd/trunk/docs/manual/bind.html.de
URL: 
http://svn.apache.org/viewvc/httpd/httpd/trunk/docs/manual/bind.html.de?rev=1851794=1851793=1851794=diff
==
--- httpd/httpd/trunk/docs/manual/bind.html.de (original)
+++ httpd/httpd/trunk/docs/manual/bind.html.de Tue Jan 22 09:53:04 2019
@@ -46,7 +46,7 @@
  ApacheKommentare
  
  
-Überblick
+Überblick 


I noticed my recent xform tried to do something like this too.


I used a recent Java 8 (1.8.0_201). It seems the change was already part 
of the checked in 2.4 docs, so there the commit diff was much smaller.
In trunk and 2.4 I both regenerated the docs using the same Java and the 
commands:


build.sh extraclean bootstrap modulelists metafiles \
 convmap typemaps validate-xml
build.sh all convmap typemaps validate-xhtml

I noticed the change above but I haven't investigated the reason. 
Especially when I saw the 2.4 was fine, so some other committers produce 
the same transformations.


Regards,

Rainer


Re: Apache 0-day / apache-uaf / use after free bugs

2019-01-22 Thread Rainer Jung

Am 22.01.2019 um 10:33 schrieb Daniel Gruno:

On 1/22/19 8:09 AM, Stefan Priebe - Profihost AG wrote:

Hi,

in twitter and other social media channels they're talking about a
current apache 0 day:
https://twitter.com/i/web/status/1087593706444730369

which wasn't handled / isn't currently fixed.

Some details are here:
https://github.com/hannob/apache-uaf

If this is true there will be exploits soon. Is there anything planned?
Does 2.4.38 fix those issues?

Greets,
Stefan



Hi Stefan, and good morning.

I figured I should write something to calm people that might be concerned.

I will reply in length in a while (coffee is needed first), it takes 
time to write a proper response that explains our processes and 
considerations with issues like this, especially when people start 
hyping the matter. Such is social media, I guess.


Until then, I will say quickly that we do not at present consider this 
something you should be alarmed about. Boring elaboration to follow in a 
while when I have compiled it :)


With regards,
Daniel, speaking as just a normal committer.


Here's the response we have compiled from Daniel, Stefan and others:

https://bz.apache.org/bugzilla/show_bug.cgi?id=63098

Regards,

Rainer


Re: [VOTE] Release httpd-2.4.38

2019-01-20 Thread Rainer Jung

Hi Dennis,

Am 21.01.2019 um 00:34 schrieb Dennis Clarke:

On 1/20/19 2:19 PM, Rainer Jung wrote:

Am 17.01.2019 um 19:49 schrieb Daniel Ruggeri:

Hi, all;
    Please find below the proposed release tarball and signatures:
https://dist.apache.org/repos/dist/dev/httpd/

I would like to call a VOTE over the next few days to release this 
candidate tarball as 2.4.38:

[X] +1: It's not just good, it's good enough!
[ ] +0: Let's have a talk.
[ ] -1: There's trouble in paradise. Here's what's wrong.


+1 to release and thanks a bunch for RM!

Summary: all OK except for

- some shutdown crashes on Solaris with MPM event when statically 
linked (already observed in 2.4.37)




That is a brutally detailed pile of work there and I am impressed and
curious.  I have done everything as 64-bit objects everywhere and wonder
if you could share what you see on Solaris 10 from :


I guess you mean for a build that shows the crashes (statically linked 
ones)? Note mine are 32 bit binaries and reallyall module set, so lots 
of dependencies. The following is from a build against OpenSSL 1.1.1a:



corv # ldd /usr/local/bin/httpd
     libpcre.so.1 =>  /usr/local/lib/libpcre.so.1
     libaprutil-1.so.0 => /usr/local/lib/libaprutil-1.so.0
     libexpat.so.1 => /usr/local/lib/libexpat.so.1
     libiconv.so.2 => /usr/local/lib/libiconv.so.2
     libapr-1.so.0 => /usr/local/lib/libapr-1.so.0
     libuuid.so.1 =>  /lib/64/libuuid.so.1
     libsendfile.so.1 =>  /lib/64/libsendfile.so.1
     librt.so.1 =>    /lib/64/librt.so.1
     libsocket.so.1 =>    /lib/64/libsocket.so.1
     libnsl.so.1 =>   /lib/64/libnsl.so.1
     libpthread.so.1 =>   /lib/64/libpthread.so.1
     libc.so.1 => /lib/64/libc.so.1
     libaio.so.1 =>   /lib/64/libaio.so.1
     libmd.so.1 =>    /lib/64/libmd.so.1
     libmp.so.2 =>    /lib/64/libmp.so.2
     libscf.so.1 =>   /lib/64/libscf.so.1
     libdoor.so.1 =>  /lib/64/libdoor.so.1
     libuutil.so.1 => /lib/64/libuutil.so.1
     libgen.so.1 =>   /lib/64/libgen.so.1
     libm.so.2 => /lib/64/libm.so.2
     /lib/sparcv9/../libm/sparcv9/libm_hwcap1.so.2
     /platform/SUNW,SPARC-Enterprise/lib/sparcv9/libc_psr.so.1


Mine is:

libjansson.so.4 => 
/path/to/install/jansson/2.12-1.solaris10.sparc/lib/libjansson.so.4
libcurl.so.4 => 
/path/to/install/curl/7.63.0-111-3.solaris10.sparc/lib/libcurl.so.4
libssl.so.1.1 => 
/path/to/install/openssl-1.1.1a-1.solaris10.sparc/lib/libssl.so.1.1
libcrypto.so.1.1 => 
/path/to/install/openssl-1.1.1a-1.solaris10.sparc/lib/libcrypto.so.1.1
libpcre.so.1 => 
/path/to/install/pcre/8.42-1.solaris10.sparc/lib/libpcre.so.1
libdistcache.so.1 => 
/path/to/install/distcache/1.5.1-9.solaris10.sparc/lib/libdistcache.so.1
libnal.so.1 => 
/path/to/install/distcache/1.5.1-9.solaris10.sparc/lib/libnal.so.1

libdl.so.1 =>/lib/libdl.so.1
libxml2.so.2 => 
/path/to/install/libxml2/2.9.9-1.solaris10.sparc/lib/libxml2.so.2

libz.so.1 => /usr/lib/libz.so.1
libbrotlienc.so.1 => 
/path/to/install/brotli/1.0.7-1.solaris10.sparc/lib/libbrotlienc.so.1
libbrotlicommon.so.1 => 
/path/to/install/brotli/1.0.7-1.solaris10.sparc/lib/libbrotlicommon.so.1

libldap.so.5 =>  /usr/lib/libldap.so.5
liblua.so.5.3 => 
/path/to/install/lua/5.3.5-1.solaris10.sparc/lib/liblua.so.5.3

libm.so.2 => /lib/libm.so.2
libaprutil-1.so.0 => 
/path/to/install/apr-util/1.6.x/1.6.1/solaris10.sparc-modular_enable-apr_1.6.5-dso_enable-expat_2.2.6-1-ldap_explicit-openssl_1.1.1a-1-shared-sqlite_3.26.0-1-bdb_6.1.19-1-mysql_6.1.11-5-oracle_11.2.0.2.0/lib/libaprutil-1.so.0
libdb-6.1.so => 
/path/to/install/berkeley_db/6.1.19-1.solaris10.sparc/lib/libdb-6.1.so

libresolv.so.2 =>/lib/libresolv.so.2
libexpat.so.1 => 
/path/to/install/expat/2.2.6-1.solaris10.sparc/lib/libexpat.so.1
libapr-1.so.0 => 
/path/to/install/apr/1.6.x/1.6.5/solaris10.sparc-dso_enable/lib/libapr-1.so.0

libuuid.so.1 =>  /lib/libuuid.so.1
libsendfile.so.1 =>  /lib/libsendfile.so.1
librt.so.1 =>/lib/librt.so.1
libsocket.so.1 =>/lib/libsocket.so.1
libnsl.so.1 =>   /lib/libnsl.so.1
libpthread.so.1 =>   /lib/libpthread.so.1
libc.so.1 => /lib/libc.so.1
libnghttp2.so.14 => 
/path/to/install/curl/7.63.0-111-3.solaris10.sparc/lib/curl-deps/libnghttp2.so.14

libsasl.so.1 =>  /usr/lib/libsasl.so.1
libmd.so.1 =>/lib/libmd.so.1
libnspr4.so =>   /usr/lib/mps/libnspr4.so
libplc4.so =>/usr/lib/mps/libplc4.so
libnss3.so =>/usr/lib/mps/libnss3.so
libssl

Re: [VOTE] Release httpd-2.4.38

2019-01-20 Thread Rainer Jung

Am 17.01.2019 um 19:49 schrieb Daniel Ruggeri:

Hi, all;
    Please find below the proposed release tarball and signatures:
https://dist.apache.org/repos/dist/dev/httpd/

I would like to call a VOTE over the next few days to release this 
candidate tarball as 2.4.38:

[X] +1: It's not just good, it's good enough!
[ ] +0: Let's have a talk.
[ ] -1: There's trouble in paradise. Here's what's wrong.


+1 to release and thanks a bunch for RM!

Summary: all OK except for

- some shutdown crashes on Solaris with MPM event when statically linked 
(already observed in 2.4.37)


Detailed report:

- Sigs and hashes OK
- contents of tarballs identical
- contents of tag and tarballs identical
  except for expected deltas

Built on

- Solaris 10 Sparc as 32 Bit Binaries
- SLES 11+12 (64 Bits)
- RHEL 6+7 (64 Bits)

For all platforms built

- with default (shared) and static modules
- with module set reallyall
- using --enable-load-all-modules
- against external APR/APU 1.6.5/1.6.1

- using external libraries
  - expat 2.2.6
  - pcre 8.42
  - lua 5.3.5 (compiled with LUA_COMPAT_MODULE)
  - distcache 1.5.1
  - libxml2 2.9.9
  - libnghttp2 1.35.1
  - brotli 1.0.7
  - curl 7.63.0
  - jansson 2.12
and
  - openssl 0.9.8zh, 1.0.2q, 1.0.2, 1.0.1e, 1.0.1i, 1.1.1, 1.1.1a

- Tool chain:
- platform gcc except on Solaris
  (gcc 8.2.0 Solaris 10)
- CFLAGS: -O2 -g -Wall -fno-strict-aliasing
  - on Solaris additionally -mpcu=v9, -D_XOPEN_SOURCE,
-D_XOPEN_SOURCE_EXTENDED=1, -D__EXTENSIONS__
and -D_XPG6

All of the 126 builds succeeded.

- compiler warnings: none

Tested for

- Solaris 10, SLES 11+12, RHEL 6+7
- MPMs prefork, worker, event
  - prefork skipped on Solaris due to the accept lock problem that
leads to timeouts and thus excessive testing times in the proxy
- default and static module builds
- log level trace8
- module set reallyall
  - for "reallyall" 128 modules plus MPMs
- Perl client bundle build against OpenSSL 1.1.1, 1.1.0i, 1.0.2p and 0.9.8zh
- OpenSSL once linked statically and once as shared library

Every OpenSSL version in the client tested with every version in the 
server, not just the same version.


The total number of test suite runs was 1318 (plus some of them still 
running, the whole suite hasn't finished yet, but enough to come up with 
a clear expectation).


The following test failures were seen:

a Crashes only on Solaris and only with event MPM and static builds.
  The crash seems to happen only at the end of a process, likely due
  to double cleanup of OpenSSL.

b Test 59 of t/modules/include.t only and always on
  Solaris.
  Not a regression
  Old analysis was:
  This is due to a bug in the test, which uses strftime()
  with a "%s" pattern that is not supported on Solaris.
  Until recently the server and the test client both returned
  verbatim "%s" and the test succeeded. After updating some
  Perl modules for the http2 tests, the perl client even
  on Solaris now supports "%s" in strftime and the test starts
  to fail. It seems we have to fix the test.

c Various tests in t/apache/expr_string.t
  Not a regression.
  Test numbers : 6, 11, 14, 17, 20, 23, 26, 29
  Happens for 39 out of about 1300 runs
  (twice on SLES 11, 3 times on Solaris 10, otherwise always on RHEL6).
  The failure is always on line 87, where the error_log contents
  are checked. Could be due to logs written to NFS.

d Test 5 in t/modules/dav.t:
  Not a regression.
  4 times, once Solaris, twice RHEL 6 and once RHEL 7
  Creation, modified and now times not in the correct order.
  This seems to be a system issue, all tests done on NFS,
  many tested on virtualized guests.

e Test 54 in t/modules/cgi.t line 232:
  Twice once Solaris
  Test checks log contents. Could be false positive due to
  logs written to NFS.

f I expect prefork on Solaris still to observe timeouts during
  proxy tests like reported for previous versions, but didn't test
  it this time due to the long test runs when the problem happens.
  I will start these runs after the other ones will have finished
  just to be able to report, whether the old problem is still
  there or has changed.

g t/modules/http2.t fails for client using OpenSSL 0.9.8(zh)
  False positive.
  Calculation for number of tests was wrong in this case,
  so test complained about 52 tests expected but 56 tests run.
  Fixed now in svn.

So apart from the shutdown crashes on Solaris no real problems. I think 
the Solaris shutdown problem is not critical, because only for event 
plus static build (plus probably APU crypto enabled). Already observed 
for 2.4.37.


gdb bt:

#0  do_rand_init_ossl_ () at crypto/rand/rand_lib.c:313
#1  0xfe8b92dc in pthread_once () from /lib/libc.so.1
#2  0xfd9800b4 in CRYPTO_THREAD_run_once (once=0xfdb32088 , 
init=0xfd9738cc ) at crypto/threads_pthread.c:113
#3  0xfd974148 in RAND_set_rand_method (meth=0x0) at 
crypto/rand/rand_lib.c:664

#4  0xfd974220 in rand_cleanup_int () at crypto/rand/rand_lib.c:355
#5  0xfd95f8c4 in 

Re: AH02268: Proxy client certificate callback: downstream server wanted client certificate but none are configured

2019-01-05 Thread Rainer Jung

Am 05.01.2019 um 15:10 schrieb Graham Leggett:

Hi all,

I am trying to connect an httpd reverse proxy to a backend tomcat, and have 
this particular hop protected by a client certificate.

The error I get is:

[Sat Jan 05 14:02:54.252552 2019] [ssl:warn] [pid 16448:tid 139929388369664] 
AH02268: Proxy client certificate callback: (jira.example.com:443) downstream 
server wanted client certificate but none are configured

Ok, so httpd is telling me that the tomcat has requested a client certificate 
(entirely true) but httpd is not configured with a client certificate.

Except httpd is configured with a client certificate, as follows:

 SSLProxyEngine on
 SSLProxyMachineCertificateFile /etc/pki/httpd/client.cert
 SSLProxyMachineCertificateChainFile /etc/pki/httpd/client.chain
 SSLProxyCACertificateFile /etc/pki/httpd/client-ca.crt
 SSLProxyVerify require
 SSLProxyVerifyDepth 3

Does this functionality work in httpd v2.4.35, or is it configured incorrectly?

(As soon as I can get this working, I would like to fix our docs to be clear 
how to do this)


Since you mention 2.4.35 explicitly, the following changelog entries 
come to my mind:


2.4.37

  *) mod_ssl: Correctly merge configurations that have client 
certificates set by SSLProxyMachineCertificate{File|Path}. [Ruediger Pluem]


2.4.36

  *) mod_ssl: Fix a regression that the configuration settings for 
verify mode and verify depth were taken from the frontend connection in 
case of connections by the proxy to the backend. PR 62769. [Ruediger Pluem]


The first got broken likely in 2.4.30, the second was reported for 
2.4.34 and was only fixed in 36, so it should be broken in 35 as well.


The first has the additional log info (r1844226):

The certificates and keys loaded during configuration time got lost 
during runtime if e.g. SSLProxyMachineCertificate{File|Path} was set on 
virtual host level and there was an SSL directive at directory level, 
e.g. SSLRequire.
This fixes a regression likely introduced in r1740928 (backported in 
r1824187).

Backport of r1844002 from trunk.

Regards,

Rainer


Re: Test framework regressions - spelling and usertrack

2018-10-22 Thread Rainer Jung

Am 22.10.2018 um 15:45 schrieb Yann Ylavic:

On Mon, Oct 22, 2018 at 3:28 PM Yann Ylavic  wrote:


On Mon, Oct 22, 2018 at 3:09 PM Jim Jagielski  wrote:


These are new from a coupla day ago:


Both tests were added a few days ago, so probably not a regression
(test issues likely).


FWIW, both pass on my Linux system.


Not here on Solaris. I think new r1844562 is the correct fix for 
usertrack.t.


New speling.t and session_cookie.t work here.

regards,

Rainer



Re: t/modules/http2.t: Run only if OpenSSL >= 1.0.0 is available

2018-10-22 Thread Rainer Jung
This seems to work nicely, committed in r1844546. Tests with old OpenSSL 
either in client or server result in TLSv1 and disable h2 tests. TLS 
test requests that result in TLSv1_2 or TLSv1_3 enable h2 tests.


Regards,

Rainer

Am 22.10.2018 um 12:37 schrieb Rainer Jung:
I wonder whether it would be easier to check whether a TLS connection 
uses TLS 1.2 or later and disable the h2 test if not.


Nevertheless the module for checking the server version might be useful, 
but here I guess checking the TLS version is more appropriate.


But that might mean, that the test runs with OpenSSL 0.9.8zh in the 
client. At least I see some TLSv1.2 reuests during the test suite run 
even when using 0.9.8zh in the client. It ever happens with that version 
in the server.


Will look into it.

Regards,

Rainer

Am 21.10.2018 um 14:28 schrieb Daniel Ruggeri:


On 10/21/2018 6:46 AM, Rainer Jung wrote:

Am 18.10.2018 um 14:23 schrieb Stefan Eissing:

Am 18.10.2018 um 14:12 schrieb Rainer Jung :

- t/modules/http2.t fails when the server is build using OpenSSL
0.9.8zh with the "Bad plan.  You planned 52 tests..." message
indicating, that h2 using TLS does not work. It happens on all
platforms, but not if the client also uses OpenSSL 0.9.8zh.

I don't know whether that is expected for old OpenSSL, so can not
judge on criticality.


AFAICT, correct me if I am wrong, OpenSSL 0.9.8 does not support
TLSv1.2 and is therefore unusable with h2. The test suite seems to be
unprepared for this scenario. I will remove it after the next
release. It is not worth fixing in its current form.


I added a check agains the test suite OpenSSL version in r1844483.

I have an aditional check for the server version available.
Unfortunately I didn't find a really easy way, so here's a small
module that one can query
(c-modules/test_ssl_version/mod_test_ssl_version.c), mostly a
shortened form of mod_test_ssl.c:

 SNIP =
#define HTTPD_TEST_REQUIRE_APACHE 2

#if CONFIG_FOR_HTTPD_TEST


 
 SetHandler test-ssl-version-lookup
 


#endif

#include "httpd.h"
#include "http_config.h"
#include "http_protocol.h"
#include "http_log.h"
#include "ap_config.h"
#include "apr_optional.h"

#if AP_MODULE_MAGIC_AT_LEAST(20040425, 0) /* simply include mod_ssl.h
if using >= 2.1.0 */

#include "mod_ssl.h"

#else
/* For use of < 2.0.x, inline the declaration: */

APR_DECLARE_OPTIONAL_FN(char *, ssl_var_lookup,
 (apr_pool_t *, server_rec *,
  conn_rec *, request_rec *,
  char *));

#endif

static APR_OPTIONAL_FN_TYPE(ssl_var_lookup) *var_lookup;

static void import_ssl_var_lookup(void)
{
 var_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
}

static int test_ssl_version_lookup(request_rec *r)
{
 char *value;

 if (strcmp(r->handler, "test-ssl-version-lookup")) {
 return DECLINED;
 }

 if (r->method_number != M_GET) {
 return DECLINED;
 }

 if (!var_lookup) {
 ap_rputs("ssl_var_lookup is not available", r);
 return OK;
 }

 value = var_lookup(r->pool, r->server,
    r->connection, r, "SSL_VERSION_LIBRARY");

 if (value && *value) {
 ap_rputs(value, r);
 }
 else {
 ap_rputs("NULL", r);
 }

 return OK;
}

static void test_ssl_version_register_hooks(apr_pool_t *p)
{
 ap_hook_handler(test_ssl_version_lookup, NULL, NULL,
APR_HOOK_MIDDLE);
 ap_hook_optional_fn_retrieve(import_ssl_var_lookup,
  NULL, NULL, APR_HOOK_MIDDLE);
}

module AP_MODULE_DECLARE_DATA test_ssl_version_module = {
 STANDARD20_MODULE_STUFF,
 NULL,  /* create per-dir    config structures */
 NULL,  /* merge  per-dir    config structures */
 NULL,  /* create per-server config structures */
 NULL,  /* merge  per-server config structures */
 NULL,  /* table of config file commands   */
 test_ssl_version_register_hooks  /* register hooks     */
};
 SNIP =

and the necessary addition to http2.t to use the module:

Index: t/modules/http2.t
===
--- t/modules/http2.t   (revision 1844483)
+++ t/modules/http2.t   (working copy)
@@ -25,6 +25,16 @@
  my $openssl_version = Net::SSLeay::OPENSSL_VERSION_NUMBER();
  if ($openssl_version < 0x1000) {
  $tls_modern = 0;
+} else {
+    Apache::TestRequest::scheme("https");
+    my $url = '/test_ssl_version_lookup';
+    my $r = GET("$url");
+    $openssl_version = $r->content;
+    print STDOUT "OpenSSL version '$openssl_version'\n";
+    # OpenSSL/0.9.8zh, OpenSSL/1.0.2p etc.
+    if ($openssl_version =~ /\/0\./) {
+    $tls_modern = 0;
+    }
  }

  Apache::TestRequest::module("http2");

What do people think? Should I apply it?

Regards,

Rainer


+1


Re: t/modules/http2.t: Run only if OpenSSL >= 1.0.0 is available

2018-10-22 Thread Rainer Jung
I wonder whether it would be easier to check whether a TLS connection 
uses TLS 1.2 or later and disable the h2 test if not.


Nevertheless the module for checking the server version might be useful, 
but here I guess checking the TLS version is more appropriate.


But that might mean, that the test runs with OpenSSL 0.9.8zh in the 
client. At least I see some TLSv1.2 reuests during the test suite run 
even when using 0.9.8zh in the client. It ever happens with that version 
in the server.


Will look into it.

Regards,

Rainer

Am 21.10.2018 um 14:28 schrieb Daniel Ruggeri:


On 10/21/2018 6:46 AM, Rainer Jung wrote:

Am 18.10.2018 um 14:23 schrieb Stefan Eissing:

Am 18.10.2018 um 14:12 schrieb Rainer Jung :

- t/modules/http2.t fails when the server is build using OpenSSL
0.9.8zh with the "Bad plan.  You planned 52 tests..." message
indicating, that h2 using TLS does not work. It happens on all
platforms, but not if the client also uses OpenSSL 0.9.8zh.

I don't know whether that is expected for old OpenSSL, so can not
judge on criticality.


AFAICT, correct me if I am wrong, OpenSSL 0.9.8 does not support
TLSv1.2 and is therefore unusable with h2. The test suite seems to be
unprepared for this scenario. I will remove it after the next
release. It is not worth fixing in its current form.


I added a check agains the test suite OpenSSL version in r1844483.

I have an aditional check for the server version available.
Unfortunately I didn't find a really easy way, so here's a small
module that one can query
(c-modules/test_ssl_version/mod_test_ssl_version.c), mostly a
shortened form of mod_test_ssl.c:

 SNIP =
#define HTTPD_TEST_REQUIRE_APACHE 2

#if CONFIG_FOR_HTTPD_TEST


     
     SetHandler test-ssl-version-lookup
     


#endif

#include "httpd.h"
#include "http_config.h"
#include "http_protocol.h"
#include "http_log.h"
#include "ap_config.h"
#include "apr_optional.h"

#if AP_MODULE_MAGIC_AT_LEAST(20040425, 0) /* simply include mod_ssl.h
if using >= 2.1.0 */

#include "mod_ssl.h"

#else
/* For use of < 2.0.x, inline the declaration: */

APR_DECLARE_OPTIONAL_FN(char *, ssl_var_lookup,
     (apr_pool_t *, server_rec *,
  conn_rec *, request_rec *,
  char *));

#endif

static APR_OPTIONAL_FN_TYPE(ssl_var_lookup) *var_lookup;

static void import_ssl_var_lookup(void)
{
     var_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
}

static int test_ssl_version_lookup(request_rec *r)
{
     char *value;

     if (strcmp(r->handler, "test-ssl-version-lookup")) {
     return DECLINED;
     }

     if (r->method_number != M_GET) {
     return DECLINED;
     }

     if (!var_lookup) {
     ap_rputs("ssl_var_lookup is not available", r);
     return OK;
     }

     value = var_lookup(r->pool, r->server,
    r->connection, r, "SSL_VERSION_LIBRARY");

     if (value && *value) {
     ap_rputs(value, r);
     }
     else {
     ap_rputs("NULL", r);
     }

     return OK;
}

static void test_ssl_version_register_hooks(apr_pool_t *p)
{
     ap_hook_handler(test_ssl_version_lookup, NULL, NULL,
APR_HOOK_MIDDLE);
     ap_hook_optional_fn_retrieve(import_ssl_var_lookup,
  NULL, NULL, APR_HOOK_MIDDLE);
}

module AP_MODULE_DECLARE_DATA test_ssl_version_module = {
     STANDARD20_MODULE_STUFF,
     NULL,  /* create per-dir    config structures */
     NULL,  /* merge  per-dir    config structures */
     NULL,  /* create per-server config structures */
     NULL,  /* merge  per-server config structures */
     NULL,  /* table of config file commands   */
     test_ssl_version_register_hooks  /* register hooks     */
};
 SNIP =

and the necessary addition to http2.t to use the module:

Index: t/modules/http2.t
===
--- t/modules/http2.t   (revision 1844483)
+++ t/modules/http2.t   (working copy)
@@ -25,6 +25,16 @@
  my $openssl_version = Net::SSLeay::OPENSSL_VERSION_NUMBER();
  if ($openssl_version < 0x1000) {
  $tls_modern = 0;
+} else {
+    Apache::TestRequest::scheme("https");
+    my $url = '/test_ssl_version_lookup';
+    my $r = GET("$url");
+    $openssl_version = $r->content;
+    print STDOUT "OpenSSL version '$openssl_version'\n";
+    # OpenSSL/0.9.8zh, OpenSSL/1.0.2p etc.
+    if ($openssl_version =~ /\/0\./) {
+    $tls_modern = 0;
+    }
  }

  Apache::TestRequest::module("http2");

What do people think? Should I apply it?

Regards,

Rainer


+1


Re: t/security/CVE-2009-3555.t fails in 2.4.37 with TLS 1.3 - also false positive?

2018-10-22 Thread Rainer Jung
Can anyone comment on the below, especially whether this test should be 
disabled when used with TLS 1.3 (modern access) and whether it is OK (a 
wrong test definition) for 1.3 to actually handle the prefix attack request?


Regards,

Rainer

Am 20.10.2018 um 08:16 schrieb Rainer Jung:
Test t/security/CVE-2009-3555.t (hardening against MITM 
SSL-renegotiation) fails in 2.4.37 when actually using TLS 1.3.


It is not that easy to use TLS 1.3 for this test. The test uses a raw 
SSL socket created by Net::SSL, but that module is outdated and does not 
support TLS 1.3.


I patched TestRequest.pm to use IO::Socket::SSL instead and can see, 
that it actually uses TLS 1.3 and the test fails at the critical last 
check. With TLS 1.2 the request that triggers a reneg but also has a 
pipelined request behind it triggers a "Connection: close" and that is 
tested in this last test. With 1.3 the server handles both the request 
that triggers the reneg as well as the pipelined on after it. That one 
fails with 400, because it does not have a host header, but it I add the 
host header, it results in a 404 not found due to the garbage URL.


What I am not sure about: CVE-2009-3555 ist mostly about a MITM attack 
for SSL renegotiations. The fix should have gone into OpenSSL itself, at 
least as far as I understand it. So it seems that our CVE-2009-3555.t 
test only checks, whether we have our workaround for non-safe OpenSSL in 
place. Because I expect TLS1.3 and any OpenSSL version supporting it not 
being affected by CVE-2009-3555, that would be a false positive as well.


Does that sound reasonable?

I will commit my TLS 1.3 patches for the test framework. I hope I 
doesn't break stuff, but it seems important to be able to run tests with 
the new protocol.


Re: [VOTE] Release httpd-2.4.37

2018-10-21 Thread Rainer Jung

Hi Dennis,

Am 22.10.2018 um 02:15 schrieb Dennis Clarke:

On 10/21/2018 08:03 PM, Rainer Jung wrote:

Am 18.10.2018 um 16:36 schrieb Daniel Ruggeri:

Hi, all;
    Please find below the proposed release tarball and signatures:
https://dist.apache.org/repos/dist/dev/httpd/

I would like to call a VOTE over the next few days to release this 
candidate tarball as 2.4.37:

[X] +1: It's not just good, it's good enough!
[ ] +0: Let's have a talk.
[ ] -1: There's trouble in paradise. Here's what's wrong.

The computed digests of the tarball up for vote are:
sha1: b0521606d1df54bb425adcdecf6348f126aa352c *httpd-2.4.37.tar.gz
sha256: 
aa97a834a32d51974be8d8a013b561e28d327387cb1da2c3c2762acd0146aabd 
*httpd-2.4.37.tar.gz





Built on

- Solaris 10 Sparc as 32 Bit Binaries


Amazing work.  I have no idea what blazing fast hardware you are
using to get this done. Special chemicals in the coffee port?


My Solaris hardware is really slow (V245) and building GCC and running 
its test suite takes a few days here as well. Buiding httpd is much faster.



I am still churning away on a fully 64-bit build and that means a
toolchain update as well as a new gcc 8.2.0 thrown in to make some
things more easy.

No signs of daylight yet.

However if it works as a 32-bit then hey should work as 64-bit? ;-)


You'll see ;)

Regards and thanks for testing,

Rainer



Re: [VOTE] Release httpd-2.4.37

2018-10-21 Thread Rainer Jung

Am 18.10.2018 um 16:36 schrieb Daniel Ruggeri:

Hi, all;
    Please find below the proposed release tarball and signatures:
https://dist.apache.org/repos/dist/dev/httpd/

I would like to call a VOTE over the next few days to release this 
candidate tarball as 2.4.37:

[X] +1: It's not just good, it's good enough!
[ ] +0: Let's have a talk.
[ ] -1: There's trouble in paradise. Here's what's wrong.

The computed digests of the tarball up for vote are:
sha1: b0521606d1df54bb425adcdecf6348f126aa352c *httpd-2.4.37.tar.gz
sha256: aa97a834a32d51974be8d8a013b561e28d327387cb1da2c3c2762acd0146aabd 
*httpd-2.4.37.tar.gz


+1 to release and thanks a ton for RM!

Summary: all OK except for

- the CVE-2009-3555.t test with OpenSSL 1.1.1
- some shutdown crashes on Solaris event when statically linked

Detailed report:

- Sigs and hashes OK
- contents of tarballs identical
- contents of tag and tarballs identical
  except for expected deltas

Built on

- Solaris 10 Sparc as 32 Bit Binaries
- SLES 11+12 (64 Bits)
- RHEL 6+7 (64 Bits)

For all platforms built

- with default (shared) and static modules
- with module set reallyall
- using --enable-load-all-modules
- against external APR/APU 1.6.5/1.6.1

- using external libraries
  - expat 2.2.6
  - pcre 8.42
  - lua 5.3.5 (compiled with LUA_COMPAT_MODULE)
  - distcache 1.5.1
  - libxml2 2.9.8
  - libnghttp2 1.33.0
  - brotli 1.0.6
  - curl 7.61.1
  - jansson 2.11
and
  - openssl 0.9.8zh, 1.0.2p, 1.0.2, 1.0.1e, 1.0.1i, 1.1.1

- Tool chain:
- platform gcc except on Solaris
  (gcc 8.2.0 Solaris 10)
- CFLAGS: -O2 -g -Wall -fno-strict-aliasing
  - on Solaris additionally -mpcu=v9, -D_XOPEN_SOURCE,
-D_XOPEN_SOURCE_EXTENDED=1, -D__EXTENSIONS__
and -D_XPG6

All of the 216 builds succeeded.

- compiler warnings: none

Tested for

- Solaris 10, SLES 11+12, RHEL 6+7
- MPMs prefork, worker, event
  - prefork skipped on Solaris due to the accept lock problem that
leads to timeouts and thus excessive testing times in the proxy
- default and static module builds
- log level trace8
- module set reallyall
  - for "reallyall" 128 modules plus MPMs
- Perl client bundle build against OpenSSL 1.1.1, 1.1.0i, 1.0.2p and 0.9.8zh
- OpenSSL linked statically and as shared library

Every OpenSSL version in the client tested with every version in the 
server, not just the same version. Client and server both with OpenSSL 
1.1.1 really resulted in TLS 1.3 being used in most SSL tests (after 
patching the test framework, all patches are committed to svn).


The total number of test suite runs was 1178.

The following test failures were seen:

a Crashes only on Solaris and only with event MPM and static builds.
  The crash seems to happen only at the end of a process, likely due
  to double cleanup of OpenSSL.

b Test 154 of t/modules/include.t only and always on
  Solaris.
  Not a regression
  Old analysis was:
  This is due to a bug in the test, which uses strftime()
  with a "%s" pattern that is not supported on Solaris.
  Until recently the server and the test client both returned
  verbatim "%s" and the test succeeded. After updating some
  Perl modules for the http2 tests, the perl client even
  on Solaris now supports "%s" in strftime and the test starts
  to fail. It seems we have to fix the test.

c Various tests in t/apache/expr_string.t
  Not a regression.
  Test numbers : 6, 11, 14, 17, 20, 23, 26, 29
  Happens for 47 out of about 1100 runs
  (once on SLES 11, once on Solaris 10, otherwise always on RHEL6).
  The failure is always on line 87, where the error_log contents
  are checked. Could be due to logs written to NFS.

d Test 5 in t/modules/dav.t:
  Not a regression.
  Only once, this time on SLES 11.
  Creation, modified and now times not in the correct order.
  This seems to be a system issue, all tests done on NFS,
  many tested on virtualized guests.

e I expect prefork on Solaris still to observe timeouts during
  proxy tests like reported for previous versions, but didn't test
  it this time due to the long test runs when the problem happens.
  I started these runs right now just to be able to report,
  whether the old problem is still there or has changed.

f t/security/CVE-2009-3555.t
  Fails in two ways:´, the first one I am unsure about the
  criticality:
  - When using OpenSSL 1.1.1 in client and server, it fails
in test 4, because the attacker request actually gets processed.
For the classic pre-1.3 handshake, there's special handling
to close the connection before the attacker request gets
handled. I am not 100% sure, but it looks to me, as if this
protection is only needed if the OpenSSL library itself is not
fixed against CVE-2009-3555 as an application side workaround.
So this should not be relevant for OpenSSL 1.1.1, and instead the
test s broken there. It would be nice if this opinion
could be confirmed by others. See the CVE-2009-3555 mail thread.
  - For other OpenSSL versions fails in test 3 

Re: error: ‘DEFAULT_REL_STATEDIR’ undeclared

2018-10-21 Thread Rainer Jung

Am 21.10.2018 um 12:58 schrieb Danesh Daroui:

Hi all,

I cannot compile the code on trunk. I get the following error when I
try to compile the code:

error: ‘DEFAULT_REL_STATEDIR’ undeclared

I bisected the mainstream using git and the erroneous commit seems to be:


---
commit 16211a8cdd52251cb7ae251e693b9053fb545e20
Author: Joe Orton 
Date:   Fri Oct 5 15:25:04 2018 +

 Define "state directory" for storing persistent child-writable state,
 with default from config.layout, configurable via DefaultStateDir.

 * server/core.c (set_state_dir, ap_state_dir_relative):
   New functions.

 * config.layout, acinclude.m4, Makefile.in, configure.in: Define
   statedir variables, drop davlockdb.

 * include/ap_config_layout.h.in: Define DEFAULT_REL_STATEDIR,
   DEFAULT_EXP_STATEDIR in place of _DAVLOCKDB.

 * include/ap_mmn.h: Bump MMN minor.


 git-svn-id:
https://svn.apache.org/repos/asf/httpd/httpd/trunk@1842929
13f79535-47bb-0310-9956-ffa450edef68
---



You may already know about the problem. So in that case is there any
fix for that?


Could it be the following bug in the generic layout, hat I just now 
fixed in r1844484:


--- httpd/httpd/trunk/config.layout (original)
+++ httpd/httpd/trunk/config.layout Sun Oct 21 12:10:09 2018
@@ -355,7 +355,7 @@
 manualdir: ${datadir}/manual
 cgidir:${datadir}/cgi-bin
 runtimedir:${localstatedir}/run
-runtimedir:${localstatedir}/lib/httpd
+statedir:  ${localstatedir}/lib/httpd
 logfiledir:${localstatedir}/log/httpd
 proxycachedir: ${localstatedir}/cache/httpd/cache-root
 

During the build, the file include/ap_config_layout.h gets generated 
from include/ap_config_layout.h.in. I guess in your case the line for 
DEFAULT_REL_STATEDIR in the generated file contains an exmpty value?


Thanks for testing trunk!

Regards,

Rainer


t/modules/http2.t: Run only if OpenSSL >= 1.0.0 is available

2018-10-21 Thread Rainer Jung

Am 18.10.2018 um 14:23 schrieb Stefan Eissing:

Am 18.10.2018 um 14:12 schrieb Rainer Jung :

- t/modules/http2.t fails when the server is build using OpenSSL 0.9.8zh with the 
"Bad plan.  You planned 52 tests..." message indicating, that h2 using TLS does 
not work. It happens on all platforms, but not if the client also uses OpenSSL 0.9.8zh.

I don't know whether that is expected for old OpenSSL, so can not judge on 
criticality.


AFAICT, correct me if I am wrong, OpenSSL 0.9.8 does not support TLSv1.2 and is 
therefore unusable with h2. The test suite seems to be unprepared for this 
scenario. I will remove it after the next release. It is not worth fixing in 
its current form.


I added a check agains the test suite OpenSSL version in r1844483.

I have an aditional check for the server version available. 
Unfortunately I didn't find a really easy way, so here's a small module 
that one can query (c-modules/test_ssl_version/mod_test_ssl_version.c), 
mostly a shortened form of mod_test_ssl.c:


 SNIP =
#define HTTPD_TEST_REQUIRE_APACHE 2

#if CONFIG_FOR_HTTPD_TEST



SetHandler test-ssl-version-lookup



#endif

#include "httpd.h"
#include "http_config.h"
#include "http_protocol.h"
#include "http_log.h"
#include "ap_config.h"
#include "apr_optional.h"

#if AP_MODULE_MAGIC_AT_LEAST(20040425, 0) /* simply include mod_ssl.h if 
using >= 2.1.0 */


#include "mod_ssl.h"

#else
/* For use of < 2.0.x, inline the declaration: */

APR_DECLARE_OPTIONAL_FN(char *, ssl_var_lookup,
(apr_pool_t *, server_rec *,
 conn_rec *, request_rec *,
 char *));

#endif

static APR_OPTIONAL_FN_TYPE(ssl_var_lookup) *var_lookup;

static void import_ssl_var_lookup(void)
{
var_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
}

static int test_ssl_version_lookup(request_rec *r)
{
char *value;

if (strcmp(r->handler, "test-ssl-version-lookup")) {
return DECLINED;
}

if (r->method_number != M_GET) {
return DECLINED;
}

if (!var_lookup) {
ap_rputs("ssl_var_lookup is not available", r);
return OK;
}

value = var_lookup(r->pool, r->server,
   r->connection, r, "SSL_VERSION_LIBRARY");

if (value && *value) {
ap_rputs(value, r);
}
else {
ap_rputs("NULL", r);
}

return OK;
}

static void test_ssl_version_register_hooks(apr_pool_t *p)
{
ap_hook_handler(test_ssl_version_lookup, NULL, NULL, APR_HOOK_MIDDLE);
ap_hook_optional_fn_retrieve(import_ssl_var_lookup,
 NULL, NULL, APR_HOOK_MIDDLE);
}

module AP_MODULE_DECLARE_DATA test_ssl_version_module = {
STANDARD20_MODULE_STUFF,
NULL,  /* create per-dirconfig structures */
NULL,  /* merge  per-dirconfig structures */
NULL,  /* create per-server config structures */
NULL,  /* merge  per-server config structures */
NULL,  /* table of config file commands   */
test_ssl_version_register_hooks  /* register hooks 
*/

};
 SNIP =

and the necessary addition to http2.t to use the module:

Index: t/modules/http2.t
===
--- t/modules/http2.t   (revision 1844483)
+++ t/modules/http2.t   (working copy)
@@ -25,6 +25,16 @@
 my $openssl_version = Net::SSLeay::OPENSSL_VERSION_NUMBER();
 if ($openssl_version < 0x1000) {
 $tls_modern = 0;
+} else {
+Apache::TestRequest::scheme("https");
+my $url = '/test_ssl_version_lookup';
+my $r = GET("$url");
+$openssl_version = $r->content;
+print STDOUT "OpenSSL version '$openssl_version'\n";
+# OpenSSL/0.9.8zh, OpenSSL/1.0.2p etc.
+if ($openssl_version =~ /\/0\./) {
+$tls_modern = 0;
+}
 }

 Apache::TestRequest::module("http2");

What do people think? Should I apply it?

Regards,

Rainer


Re: Test suite and OpenSSL 1.1.1

2018-10-20 Thread Rainer Jung
Plus r1844425 which simplifies TestRequest.pm since IO::Socket::SSL has 
a working getline().


Am 20.10.2018 um 09:59 schrieb Rainer Jung:
I now also added r1844396 to allow setting the CA for peer cert 
verification and used it in echo.t and nttp-like.t to unbreak their ssl 
testing (r1844397).


I didn't find more uses of the raw sockets.

Regards,

Rainer

Am 20.10.2018 um 08:47 schrieb Rainer Jung:
To make the raw TLS socket tests work I added r1844393. Both, r1844389 
and r1844393 are part of the /perl/Apache-Test/trunk/ external which 
gets pulled into our test framework.


Am 20.10.2018 um 06:28 schrieb Rainer Jung:

Am 19.10.2018 um 23:31 schrieb Yann Ylavic:
Could not make the test suite framework work with 1.1.1 (cpan -u 
didn't help).

Although the ssl tests report SUCCESS, httpd actually timeouts on
SSL_peek() (as already reported).


Indeed I checked my test suite logs and until now all tests only used 
TLS 1.2. But what works for me now with TLS 1.3 is:


- small fix in TestSSLCA.pm (r1844389), otherwise the geneated 
t/conf/ssl/ssl.conf always contains "SSLProtocol all -TLSv1.3" 
instead of "all" (unless you specifiy -sslproto explicitly).


- Net::SSLeay 1.86_06 tag from Github 
https://github.com/radiator-software/p5-net-ssleay.git. Added "-ldl 
-pthread" to OTHERLDFLAGS in Makefile. It contains the plumbing 
needed for some new 1.1.1 APIs.


- IO/Socket/SSL.pm recent version 2.060 plus patch 
https://github.com/noxxi/p5-io-socket-ssl/commit/e96b1c9e394011de4ee181cfa42b8021796bf7d4.patch 
(probably not needed) plus anti-hang patch to call 
Net::SSLeay::CTX_set_post_handshake_auth()


--- IO/Socket/SSL.pm.orig  2018-08-15 18:03:29.0 +
+++ IO/Socket/SSL.pm   2018-09-19 16:37:46.450281000 +
@@ -2594,6 +2594,10 @@
 "Failed to load key from file (no PEM or DER)");
 }

+    if ($havecert && $havekey && 
Net::SSLeay::OPENSSL_VERSION_NUMBER() >= 0x1010100f) {

+    Net::SSLeay::CTX_set_post_handshake_auth($ctx, 1);
+    }
+
 # replace arg_hash with created context
 $ctx{$host} = $ctx;
  }

The PHA patch was stolen from Joe's explanation of the PHA issue.

With this setup, I can see some TLSv1.3 entries in the 
t/logs/ssl_request_log. For instance when running t/ssl/varlookup.t.


Regards,

Rainer


Re: Test suite and OpenSSL 1.1.1

2018-10-20 Thread Rainer Jung

Am 20.10.2018 um 13:26 schrieb Christophe JAILLET:

Le 20/10/2018 à 11:00, Rainer Jung a écrit :

Am 20.10.2018 um 10:27 schrieb Christophe JAILLET:

Le 20/10/2018 à 09:56, Rainer Jung a écrit :

Am 20.10.2018 um 09:39 schrieb Christophe JAILLET:

Le 20/10/2018 à 06:28, Rainer Jung a écrit :

Am 19.10.2018 um 23:31 schrieb Yann Ylavic:
Could not make the test suite framework work with 1.1.1 (cpan -u 
didn't help).

Although the ssl tests report SUCCESS, httpd actually timeouts on
SSL_peek() (as already reported).


Indeed I checked my test suite logs and until now all tests only 
used TLS 1.2. But what works for me now with TLS 1.3 is:


- small fix in TestSSLCA.pm (r1844389), otherwise the geneated 
t/conf/ssl/ssl.conf always contains "SSLProtocol all -TLSv1.3" 
instead of "all" (unless you specifiy -sslproto explicitly).




I've just updated the test framework.
make clean
t/TEST
--> ssl.conf rebuilt

But I still have:
    SSLProtocol all -TLSv1.3


I didn't manage to rebuild ssl.conf using make, but what I did to 
rebuild was a "t/TEST -v -configure" and to make sure I removed the 
ssl.conf file before running that command. This resulted in a new 
file with "all" in it.


Please also double check, that TestSSLCA.pm contains the line "use 
Net::SSLeay;".


Does it work with that recipe?

Thanks and regards,



use Net::SSLeay;
is there.


Comment added in ssl.conf.in gets reflected in ssl.conf, so it is 
rebuilt.



t/TEST -v -configure
[warning] setting ulimit to allow core files
ulimit -c unlimited; /usr/bin/perl 
/home/tititou36/svn_test_framework/t/TEST -v -configure

[warning] cleaning out current configuration
[warning] skipping rebuild of c-modules; run t/TEST -clean to force
[warning] skipping regeneration of SSL CA; run t/TEST -clean to force
make: rien à faire pour « all ».
[warning] reconfiguration done

But SSLProtocol all -TLSv1.3 is still there.


t/TEST -clean
doesn't help either.


The check, wheher "all" or "all -TLSv1.3" is put into the file is done 
in TestSSLCA.pm. The code there checks the following, which you can 
also check in a test script to see, which condition fails:


Apache::Test::normalize_vstring(Apache::Test::version()) >=
Apache::Test::normalize_vstring("1.1.1")

and

defined(::SSLeay::CTX_set_post_handshake_auth)

The first looks for the OpenSSL version caused by your test framework, 
the second checks, whether Net::SSLeay is current (actually at least 
developer snapshot 1.86_06). Both is needed to make TLS 1.3 work in 
the test framework.


To check standalone you can use a script like this:

=== SNIP ===

#!/usr/bin/perl

use strict;
use Net::SSLeay;
use IO::Socket::SSL;
use Apache::Test;
use Apache::TestSSLCA;

my $version = Apache::TestSSLCA::version();
print "OpenSSL version: $version\n";
print "Normalized OpenSSL version: " .
    Apache::Test::normalize_vstring($version) . "\n";
print "Normalized 1.1.1 version: " .
    Apache::Test::normalize_vstring("1.1.1") . "\n";
print "Net::SSLeay::VERSION: $Net::SSLeay::VERSION\n";
print "IO::Socket::SSL::VERSION: $IO::Socket::SSL::VERSION\n";
print "Net::SSLeay::CTX_set_post_handshake_auth available: " .
    (defined(::SSLeay::CTX_set_post_handshake_auth) ?
    "true" : "false") . "\n";
my $tls13 = (Apache::Test::normalize_vstring($version) >=
    Apache::Test::normalize_vstring("1.1.1")) &&
    defined(::SSLeay::CTX_set_post_handshake_auth);
print "TLSv1.3 support: " . ($tls13 ? "true" : "false") . "\n";

=== SNIP ===

To run it you must also provide the path to the test framework and if 
you have installed the additional moduls needed by the framework in 
some special place, you must also provide this one, both via "-I" flag:


perl -I /path/to/bundle/lib/perl5 -I /path/to/Apache-Test/lib test.pl

When I run this I get:

OpenSSL version: 1.1.1
Normalized OpenSSL version: 001001001
Normalized 1.1.1 version: 001001001
Net::SSLeay::VERSION: 1.86_06
IO::Socket::SSL::VERSION: 2.060
Net::SSLeay::CTX_set_post_handshake_auth available: true
TLSv1.3 support: true

Most likely your version of Net::SSLeay is to old.

In adition, once the framework detects TLSv1.3 correct, you also need 
IO::Socket::SSL 2.060 plus the one patch for its SSL.pm that I 
mentioned at the beginning of this thread.


Regards,

Rainer


OpenSSL version: 1.1.1
Normalized OpenSSL version: 001001001
Normalized 1.1.1 version: 001001001
Net::SSLeay::VERSION: 1.85 <-
IO::Socket::SSL::VERSION: 2.060
Net::SSLeay::CTX_set_post_handshake_auth available: false
TLSv1.3 support: false <-

When I try to update it using perl -MCPAN -e ..., I get:

Net::SSLeay is up to date (1.85).
which is in line with https://metacpan.org/pod/Net::SSLeay


I will have to wait fo

  1   2   3   4   5   6   7   8   9   10   >