Re: [VOTE] Release apr-1.7.5-rc1 as apr-1.7.5

[Rainer Jung]

Hi all,

Am 21.08.24 um 00:08 schrieb Eric Covener:

Hi all,

Please find below the proposed release tarball and signatures:

https://dist.apache.org/repos/dist/dev/apr/

I would like to call a VOTE over the next few days to release
this candidate tarball apr-1.7.5-rc1 as 1.7.5:
[X] +1: It's not just good, it's good enough!
[ ] +0: Let's have a talk.
[ ] -1: There's trouble in paradise. Here's what's wrong.

The computed digests of the tarball up for vote are:


The SVN candidate source is found at tags/1.7.5-rc1-candidate.


+1 for release.

Tested on RHEL 6, 7, 8 and 9, on SLES 11, 12 and 15 all x86_64 and on 
Solaris 10 Sparc.


make check was fine, also built some apr-util variants against it 
including successful make check. Finally built a variation of httpd 
2.4.62 configurations on these platforms and ran the unit tests. All fine.


Thanks for RM!

Rainer

Re: [VOTE] Release apr-1.7.5-rc1 as apr-1.7.5

[Rainer Jung]

Hi Eric,

Am 21.08.24 um 00:08 schrieb Eric Covener:

Hi all,

Please find below the proposed release tarball and signatures:

https://dist.apache.org/repos/dist/dev/apr/

I would like to call a VOTE over the next few days to release
this candidate tarball apr-1.7.5-rc1 as 1.7.5:
[ ] +1: It's not just good, it's good enough!
[ ] +0: Let's have a talk.
[ ] -1: There's trouble in paradise. Here's what's wrong.

The computed digests of the tarball up for vote are:


The SVN candidate source is found at tags/1.7.5-rc1-candidate.


File apr-1.7.5-rc1-candidate-win32-src.zip seems to be missing. There 
are asc and sha256 files for it but not the file itself.


The Announcement files habe changed their name from 1.7 to 1.x and the 
CHANGES file is missing, so we might double check, that the files are 
there and have the expected names, once the release gets published.


Now on to testing...

Thanks and regards,

Rainer

Re: svn commit: r1916666 - in /apr/apr/branches/1.8.x: NOTICE include/apr_version.h

[Rainer Jung]

Hi Roy,

I adjusted the Copyright notice for APR and APR-Util in the form the web 
server uses it. I hope it is OK now? Otherwise please let me know.


Thanks and best regards,

Rainer

Am 30.03.24 um 21:54 schrieb Roy T. Fielding:

On Mar 30, 2024, at 1:40 PM, rj...@apache.org wrote:


Author: rjung
Date: Sat Mar 30 20:40:29 2024
New Revision: 191

URL: http://svn.apache.org/viewvc?rev=191&view=rev
Log:
Update copyright year

Modified:
apr/apr/branches/1.8.x/NOTICE
apr/apr/branches/1.8.x/include/apr_version.h

Modified: apr/apr/branches/1.8.x/NOTICE
URL: 
http://svn.apache.org/viewvc/apr/apr/branches/1.8.x/NOTICE?rev=191&r1=1916665&r2=191&view=diff
==
--- apr/apr/branches/1.8.x/NOTICE (original)
+++ apr/apr/branches/1.8.x/NOTICE Sat Mar 30 20:40:29 2024
@@ -1,5 +1,5 @@
Apache Portable Runtime
-Copyright (c) 2000-2023 The Apache Software Foundation.
+Copyright (c) 2000-2024 The Apache Software Foundation.

This product includes software developed at
The Apache Software Foundation (http://www.apache.org/).

Modified: apr/apr/branches/1.8.x/include/apr_version.h
URL: 
http://svn.apache.org/viewvc/apr/apr/branches/1.8.x/include/apr_version.h?rev=191&r1=1916665&r2=191&view=diff
==
--- apr/apr/branches/1.8.x/include/apr_version.h (original)
+++ apr/apr/branches/1.8.x/include/apr_version.h Sat Mar 30 20:40:29 2024
@@ -38,7 +38,7 @@
  */


-#define APR_COPYRIGHT "Copyright (c) 2000-2023 The Apache Software " \
+#define APR_COPYRIGHT "Copyright (c) 2000-2024 The Apache Software " \
   "Foundation or its licensors, as applicable."

/* The numeric compile-time version constants. These constants are the



This is an old format for the copyright notice. It should be the same
as for httpd (which hasn't been updated in a while).  IOW,

Copyright 2024 The Apache Software Foundation.

and

#define APR_COPYRIGHT "Copyright 2024 The Apache Software Foundation."

because this notice is for the package as a whole and the other bits
make it an invalid/ambiguous notice.

Roy

Out of tree build broken for APR 1.7.x and 1.8.x

[Rainer Jung]

More precisely "make check" is broken for out of tree builds, although 
the core reason is not restricted to "make check". It was observed after 
r1909334 / r1909335, which introduce a dependency on 
arch/unix/apr_private.h to testpoll.c.


The header file arch/unix/apr_private.h (build tree) is generated from 
arch/unix/apr_private.h.in (source tree). But is contains


#include "../apr_private_common.h"

which will be looked up due to the relative path in the build tree, but 
actually is located in the source tree.


Currently arch/unix/apr_private.h is only used in testpoll.h, therefore 
the problem only shows up during make check. But in fact 
arch/unix/apr_private.h is broken for any potential use.


This problem is in 1.7.x and 1.8.x. I did not check apr trunk.

Unfortunately I have no idea how to make autoheader to use

#include "arch/apr_private_common.h"

instead of

#include "../apr_private_common.h"

in the generated arch/unix/apr_private.h.in.

Maybe someone has an idea?

Thanks and regards,

Rainer

Re: svn commit: r1906889 - in /apr/apr/trunk: ./ build/ include/ include/arch/win32/ test/ threadproc/beos/ threadproc/netware/ threadproc/os2/ threadproc/unix/ threadproc/win32/

[Rainer Jung]

Am 21.03.23 um 16:46 schrieb Yann Ylavic:

On Tue, Mar 21, 2023 at 2:24 PM Yann Ylavic  wrote:


Is there something I can #ifdef (or check at configure/run time) for
SLELS11 or the kernel? This would document in the test that a minimal
timeout is usually expected, but on this platform/kernel. Or I'll use
190ms for everyone, which looks "unfair".


Does r1908616 work for you?


Thanks again, yes that works.


Regards;
Yann.

Re: svn commit: r1908503 - in /apr/apr/trunk: build/crypto.m4 crypto/apr_crypto_openssl.c

[Rainer Jung]

Thanks Yann, looks good here.

Am 27.03.23 um 15:43 schrieb Yann Ylavic:

Hi Rainer;

On Mon, Mar 27, 2023 at 2:02 PM Rainer Jung  wrote:


Am 18.03.23 um 17:33 schrieb yla...@apache.org:


@@ -113,15 +112,13 @@ AC_DEFUN([APU_CHECK_CRYPTO_OPENSSL], [
 AC_MSG_NOTICE(checking for openssl in $withval)
 AC_CHECK_HEADERS(openssl/x509.h, [openssl_have_headers=1])
 AC_CHECK_LIB(crypto, EVP_CIPHER_CTX_new, openssl_have_libs=1)
-  if test "$openssl_have_headers" != "0" && test "$openssl_have_libs" != 
"0"; then
+  if test "$openssl_have_headers" = "1" && test "$openssl_have_libs" = 
"1"; then
   apu_have_openssl=1
-APR_ADDTO(LDFLAGS, [-L$withval/lib])
-APR_ADDTO(INCLUDES, [-I$withval/include])

  > fi
  > -
  > -  AC_CHECK_DECLS([EVP_PKEY_CTX_new, OPENSSL_init_crypto], [], [],
  > - [#include ])
  > -
  > +fi
  > +if test "$apu_have_openssl" = "1"; then
  > +AC_CHECK_LIB(crypto, OPENSSL_init_crypto)
  > +AC_CHECK_FUNCS([OPENSSL_init_crypto])
  >   fi

Hmmm, for me this dropped "APR_ADDTO(INCLUDES,..." breaks the apr trunk
build. The path to the OpenSSL header files gets no longer added to any
INCLUDES variable for the build.


Ah yes, I thought both LDFLAGS and INCLUDES were already set above but
it is LDFLAGS and CPPFLAGS actually.
Restored in r1908749.

Thanks!
Yann.

Re: svn commit: r1908503 - in /apr/apr/trunk: build/crypto.m4 crypto/apr_crypto_openssl.c

[Rainer Jung]

Hi Yann,

Am 18.03.23 um 17:33 schrieb yla...@apache.org:

Author: ylavic
Date: Sat Mar 18 16:33:08 2023
New Revision: 1908503

URL: http://svn.apache.org/viewvc?rev=1908503&view=rev
Log:
crypto_openssl: Fix configure/detection of OPENSSL_init_crypto()

Modified:
 apr/apr/trunk/build/crypto.m4
 apr/apr/trunk/crypto/apr_crypto_openssl.c

Modified: apr/apr/trunk/build/crypto.m4
URL: 
http://svn.apache.org/viewvc/apr/apr/trunk/build/crypto.m4?rev=1908503&r1=1908502&r2=1908503&view=diff
==
--- apr/apr/trunk/build/crypto.m4 (original)
+++ apr/apr/trunk/build/crypto.m4 Sat Mar 18 16:33:08 2023

...


@@ -113,15 +112,13 @@ AC_DEFUN([APU_CHECK_CRYPTO_OPENSSL], [
AC_MSG_NOTICE(checking for openssl in $withval)
AC_CHECK_HEADERS(openssl/x509.h, [openssl_have_headers=1])
AC_CHECK_LIB(crypto, EVP_CIPHER_CTX_new, openssl_have_libs=1)
-  if test "$openssl_have_headers" != "0" && test "$openssl_have_libs" != 
"0"; then
+  if test "$openssl_have_headers" = "1" && test "$openssl_have_libs" = 
"1"; then
  apu_have_openssl=1
-APR_ADDTO(LDFLAGS, [-L$withval/lib])
-APR_ADDTO(INCLUDES, [-I$withval/include])

> fi
> -
> -  AC_CHECK_DECLS([EVP_PKEY_CTX_new, OPENSSL_init_crypto], [], [],
> - [#include ])
> -
> +fi
> +if test "$apu_have_openssl" = "1"; then
> +AC_CHECK_LIB(crypto, OPENSSL_init_crypto)
> +AC_CHECK_FUNCS([OPENSSL_init_crypto])
>   fi

Hmmm, for me this dropped "APR_ADDTO(INCLUDES,..." breaks the apr trunk 
build. The path to the OpenSSL header files gets no longer added to any 
INCLUDES variable for the build.


For 1.7.x and 1.6.x you committed it differently:

-APR_ADDTO(APRUTIL_LDFLAGS, [-L$withval/lib])
-APR_ADDTO(APRUTIL_INCLUDES, [-I$withval/include])
+APR_ADDTO(APRUTIL_INCLUDES, [$openssl_CPPFLAGS])
+APR_ADDTO(APRUTIL_LDFLAGS, [$openssl_LDFLAGS])

So maybe adding back APR_ADDTO for INCLUDES (and LDFLAGS?) but with the 
new values would be the right thing? Before the change, the addition to 
INCLUDES get copied over to EXTRA_INCLUDES and ends up in 
build/apr_rules.mk.


Best regards,

Rainer

Re: svn commit: r1906889 - in /apr/apr/trunk: ./ build/ include/ include/arch/win32/ test/ threadproc/beos/ threadproc/netware/ threadproc/os2/ threadproc/unix/ threadproc/win32/

[Rainer Jung]

Hi Yann,

Am 17.03.23 um 15:11 schrieb Yann Ylavic:

On Fri, Mar 17, 2023 at 2:30 PM Rainer Jung  wrote:


Am 17.03.23 um 14:12 schrieb Yann Ylavic:

On Fri, Mar 17, 2023 at 11:20 AM Rainer Jung  wrote:


But: testpoll fails:

testpoll:  Line 897: apr_pollset_poll() didn't sleep

Unfortunately I don't know when it started. Any idea, what I should
investigate?

All this is on SLES11, haven't tried the recent APR trunk with newer
Linuxes, but r1908005 worked on them including testpoll tests.


I suppose SLES11 is using epoll implementation, so r1902236 and
r1902258 may help.
Does your APR contain those?

Regards;
Yann.


Yes, it contained everything until r1908442.


Hm OK, these commits were meant to address that precisely..
It looks like SLES has some special epoll_wait() implementation that
can return before the timeout. Could you printf the t2 - t1 diff in
the justsleep() test after apr_pollset_poll()? It could be interesting
to know if it's almost 200ms or close to zero (the latter could mean
epoll_wait() does not block if there is no fd in the pollset, which is
the case in this test).


The observed time delta on this platform is between 197 and 198.5 ms 
instead of somewhat above 200ms. I should note it is an old patch level 
of SLES 11 using kernel 2.6.32.12-0.7.


There are some old informations about the rounding of the timeout:

https://linux-fsdevel.vger.kernel.narkive.com/9jp0o6FA/bug-epoll-wait-timeout-is-shorter-than-requested

On this system HZ is 100, so a jiffie is 10ms and probably this means it 
should suffice to accept 200-10=190ms as the minimum sleep time.


Best regards,

Rainer

Re: svn commit: r1906889 - in /apr/apr/trunk: ./ build/ include/ include/arch/win32/ test/ threadproc/beos/ threadproc/netware/ threadproc/os2/ threadproc/unix/ threadproc/win32/

[Rainer Jung]

Am 17.03.23 um 14:12 schrieb Yann Ylavic:

On Fri, Mar 17, 2023 at 11:20 AM Rainer Jung  wrote:


But: testpoll fails:

testpoll:  Line 897: apr_pollset_poll() didn't sleep

Unfortunately I don't know when it started. Any idea, what I should
investigate?

All this is on SLES11, haven't tried the recent APR trunk with newer
Linuxes, but r1908005 worked on them including testpoll tests.


I suppose SLES11 is using epoll implementation, so r1902236 and
r1902258 may help.
Does your APR contain those?

Regards;
Yann.


Yes, it contained everything until r1908442.

configure output contains:

ac_cv_define_POLLIN=yes
ac_cv_func_poll=yes
ac_cv_header_poll_h=yes
ac_cv_header_sys_poll_h=yes
apr_cv_epoll=yes
apr_cv_epoll_create1=yes
#define HAVE_POLL 1
#define HAVE_EPOLL 1
#define HAVE_EPOLL_CREATE1 1
#define HAVE_POLL_H 1
#define HAVE_SYS_POLL_H 1
#define HAVE_POLLIN 1
#define WAITIO_USES_POLL 1

Thanks and regards,

Rainer

Re: svn commit: r1906889 - in /apr/apr/trunk: ./ build/ include/ include/arch/win32/ test/ threadproc/beos/ threadproc/netware/ threadproc/os2/ threadproc/unix/ threadproc/win32/

[Rainer Jung]

Thanks Yann, configure now detects the missing function.

But: testpoll fails:

testpoll:  Line 897: apr_pollset_poll() didn't sleep

Unfortunately I don't know when it started. Any idea, what I should 
investigate?


All this is on SLES11, haven't tried the recent APR trunk with newer 
Linuxes, but r1908005 worked on them including testpoll tests.


Thanks and regards,

Rainer

Am 16.03.23 um 16:27 schrieb Rainer Jung:

It should, I will try later and report back. Thanks!

Am 16.03.23 um 15:06 schrieb Yann Ylavic:

On Thu, Mar 16, 2023 at 2:53 PM Ivan Zhakov  wrote:


I have very little knowledge about autoconf stuff. The autoconf part 
was suggest by Yann in [1]


Yann: do you have any ideas why it does not work?


Does r1908438 fix it for you Rainer?

Regards;
Yann.

Re: svn commit: r1908433 - in /apr/apr/trunk: crypto/apr_crypto_openssl.c test/testcrypto.c

[Rainer Jung]

DIGEST;
  }
-
  }
  
-crypto_digest_cleanup(digest);

-
-return status;
-
-}
-default: {
-return APR_EINVAL;
+break;
  }
+default:
+status = APR_EINVAL;
  }
  
+return status;

  }
  
  static apr_status_t crypto_digest(


Modified: apr/apr/trunk/test/testcrypto.c
URL: 
http://svn.apache.org/viewvc/apr/apr/trunk/test/testcrypto.c?rev=1908433&r1=1908432&r2=1908433&view=diff
==
--- apr/apr/trunk/test/testcrypto.c (original)
+++ apr/apr/trunk/test/testcrypto.c Thu Mar 16 12:43:17 2023
@@ -883,6 +883,10 @@ static void crypto_block_cross(abts_case
  
  f1 = make(tc, pool, driver1);

  f2 = make(tc, pool, driver2);
+if (!f1 || !f2) {
+return;
+}
+
  key1 = passphrase(tc, pool, driver1, f1, type, mode, doPad, description);
  key2 = passphrase(tc, pool, driver2, f2, type, mode, doPad, description);
  
@@ -978,6 +982,9 @@ static void crypto_cross_hash(abts_case
  
  f1 = make(tc, pool, driver1);

  f2 = make(tc, pool, driver2);
+if (!f1 || !f2) {
+return;
+}
  
  key7 = keyhash(tc, pool, driver1, f1, digest, description);

  key8 = keyhash(tc, pool, driver2, f2, digest, description);
@@ -1043,6 +1050,9 @@ static void crypto_cross_sign(abts_case
  
  f1 = make(tc, pool, driver1);

  f2 = make(tc, pool, driver2);
+if (!f1 || !f2) {
+return;
+}
  
  key7 = keyhmac(tc, pool, driver1, f1, digest, type, mode, doPad, secretLen,

  description);
@@ -1102,6 +1112,10 @@ static void test_crypto_key_openssl(abts
  driver = get_openssl_driver(tc, pool);
  
  f = make(tc, pool, driver);

+if (!f) {
+return;
+}
+
  keysecret(tc, pool, driver, f, APR_KEY_AES_256, APR_MODE_CBC, 1, 32,
  "KEY_AES_256/MODE_CBC");
  apr_pool_destroy(pool);
@@ -1123,6 +1137,10 @@ static void test_crypto_key_nss(abts_cas
  driver = get_nss_driver(tc, pool);
  
  f = make(tc, pool, driver);

+if (!f) {
+return;
+}
+
  keysecret(tc, pool, driver, f, APR_KEY_AES_256, APR_MODE_CBC, 1, 32,
  "KEY_AES_256/MODE_CBC");
  apr_pool_destroy(pool);
@@ -1144,6 +1162,10 @@ static void test_crypto_key_commoncrypto
  driver = get_commoncrypto_driver(tc, pool);
  
  f = make(tc, pool, driver);

+if (!f) {
+return;
+}
+
  keysecret(tc, pool, driver, f, APR_KEY_AES_256, APR_MODE_CBC, 1, 32,
  "KEY_AES_256/MODE_CBC");
  apr_pool_destroy(pool);
@@ -2112,6 +2134,10 @@ static void test_crypto_get_block_key_ty
  if (driver) {
  
  f = make(tc, pool, driver);

+if (!f) {
+return;
+}
+
  apr_crypto_get_block_key_types(&types, f);
  
  key_3des_192 = apr_hash_get(types, "3des192", APR_HASH_KEY_STRING);

@@ -2157,6 +2183,10 @@ static void test_crypto_get_block_key_ty
  if (driver) {
  
  f = make(tc, pool, driver);

+if (!f) {
+return;
+}
+
  apr_crypto_get_block_key_types(&types, f);
  
  key_3des_192 = apr_hash_get(types, "3des192", APR_HASH_KEY_STRING);

@@ -2202,6 +2232,10 @@ static void test_crypto_get_block_key_ty
  if (driver) {
  
  f = make(tc, pool, driver);

+if (!f) {
+return;
+}
+
  apr_crypto_get_block_key_types(&types, f);
  
  key_3des_192 = apr_hash_get(types, "3des192", APR_HASH_KEY_STRING);

@@ -2245,6 +2279,10 @@ static void test_crypto_get_block_key_mo
  if (driver) {
  
  f = make(tc, pool, driver);

+if (!f) {
+return;
+}
+
  apr_crypto_get_block_key_modes(&modes, f);
  
  mode_ecb = apr_hash_get(modes, "ecb", APR_HASH_KEY_STRING);

@@ -2280,6 +2318,10 @@ static void test_crypto_get_block_key_mo
  if (driver) {
  
  f = make(tc, pool, driver);

+if (!f) {
+return;
+}
+
  apr_crypto_get_block_key_modes(&modes, f);
  
  mode_ecb = apr_hash_get(modes, "ecb", APR_HASH_KEY_STRING);

@@ -2315,6 +2357,10 @@ static void test_crypto_get_block_key_mo
  if (driver) {
  
  f = make(tc, pool, driver);

+if (!f) {
+return;
+}
+
  apr_crypto_get_block_key_modes(&modes, f);
  
  mode_ecb = apr_hash_get(modes, "ecb", APR_HASH_KEY_STRING);





--
kippdata
informationstechnologie GmbH   Tel: 0228 98549 -0
Bornheimer Str. 33aFax: 0228 98549 -50
53111 Bonn www.kippdata.de

HRB 8018 Amtsgericht Bonn / USt.-IdNr. DE 196 457 417
Geschäftsführer: Dr. Thomas Höfer, Rainer Jung, Sven Maurmann

Re: svn commit: r1906889 - in /apr/apr/trunk: ./ build/ include/ include/arch/win32/ test/ threadproc/beos/ threadproc/netware/ threadproc/os2/ threadproc/unix/ threadproc/win32/

[Rainer Jung]

It should, I will try later and report back. Thanks!

Am 16.03.23 um 15:06 schrieb Yann Ylavic:

On Thu, Mar 16, 2023 at 2:53 PM Ivan Zhakov  wrote:


I have very little knowledge about autoconf stuff. The autoconf part was 
suggest by Yann in [1]

Yann: do you have any ideas why it does not work?


Does r1908438 fix it for you Rainer?

Regards;
Yann.

Re: svn commit: r1906889 - in /apr/apr/trunk: ./ build/ include/ include/arch/win32/ test/ threadproc/beos/ threadproc/netware/ threadproc/os2/ threadproc/unix/ threadproc/win32/

[Rainer Jung]

Hello Ivan,

Am 21.01.23 um 17:13 schrieb i...@apache.org:

Author: ivan
Date: Sat Jan 21 16:13:27 2023
New Revision: 1906889

URL: http://svn.apache.org/viewvc?rev=1906889&view=rev
Log:
Merge thread-name branch (PR 60587) [1]:
* Introduce apr_thread_name_set() and apr_thread_name_get().

[1] https://bz.apache.org/bugzilla/show_bug.cgi?id=60587
[2] https://lists.apache.org/thread/z24logzc6v8tc0p2q3375cc10qo9y5yw

Modified:
 apr/apr/trunk/   (props changed)
 apr/apr/trunk/build/apr_threads.m4
 apr/apr/trunk/configure.in
 apr/apr/trunk/include/apr_thread_proc.h
 apr/apr/trunk/include/arch/win32/apr_arch_misc.h
 apr/apr/trunk/test/testthread.c
 apr/apr/trunk/threadproc/beos/thread.c
 apr/apr/trunk/threadproc/netware/thread.c
 apr/apr/trunk/threadproc/os2/thread.c
 apr/apr/trunk/threadproc/unix/thread.c
 apr/apr/trunk/threadproc/win32/thread.c

Propchange: apr/apr/trunk/
--
   Merged /apr/apr/branches/thread-name:r1902296-1906888

Modified: apr/apr/trunk/build/apr_threads.m4
URL: 
http://svn.apache.org/viewvc/apr/apr/trunk/build/apr_threads.m4?rev=1906889&r1=1906888&r2=1906889&view=diff
==
--- apr/apr/trunk/build/apr_threads.m4 (original)
+++ apr/apr/trunk/build/apr_threads.m4 Sat Jan 21 16:13:27 2023
@@ -312,3 +312,27 @@ elif test "$apr_cv_mutex_robust_shared"
   [Define if non-posix/portable cross-process robust mutexes are 
available])
  fi
  ])
+
+
+dnl Check for pthread_setname_np
+dnl Note: Only detects two-arg version
+AC_DEFUN([APR_CHECK_PTHREAD_SETNAME_NP], [
+AC_CACHE_CHECK([for pthread_setname_np support],
+[apr_cv_pthread_setname_np], [
+AC_TRY_COMPILE([
+#include 
+],[
+pthread_t td = pthread_self();
+pthread_setname_np(td, "name");
+],[
+apr_cv_pthread_setname_np=yes
+],[
+apr_cv_pthread_setname_np=no
+])])
+
+if test "$apr_cv_pthread_setname_np" = "yes"; then
+   AC_DEFINE([HAVE_PTHREAD_SETNAME_NP], 1,
+ [Define if pthread_setname_np is available])
+fi
+])dnl
+


Thsi configure check does *not* fail for me on SLES 11 where 
pthread_[gs]ername_np is not available. In config.log I find


warning: implicit declaration of function ‘pthread_setname_np’

but configure procees with a "yes". The buidl then succeeds, but the 
library is not usable due to the unresolvable symbol.


Best regards,

Rainer

Minor nag: APU build warnings (RHEL 8 plus Solaris)

[Rainer Jung]

During build and test for the 1.6.2-rc2 release vote I noticed the 
following warnings. None is new, they were already present at least for 
1.6.1:


Only Solaris

xlate/xlate.c:120:38: warning: passing argument 2 of 'iconv' from 
incompatible pointer type [-Wincompatible-pointer-types]
xlate/xlate.c:343:42: warning: passing argument 2 of 'iconv' from 
incompatible pointer type [-Wincompatible-pointer-types]



RHEL 8+Solaris (test code):

test/testbuckets.c:108:13: warning: passing argument 1 to 
restrict-qualified parameter aliases with argument 3 [-Wrestrict]
test/testbuckets.c:108:21: warning: ': result match' directive writing 
14 bytes into a region of size between 1 and 200 [-Wformat-overflow=]
test/testbuckets.c:108:5: note: 'sprintf' output between 15 and 214 
bytes into a destination of size 200


And then we have the OpenSSL 3 deprecations in cvrypto:

ENGINE *ENGINE_by_id(const char *id);
int ENGINE_finish(ENGINE *e);
int ENGINE_free(ENGINE *e);
int ENGINE_init(ENGINE *e);
int ENGINE_register_all_complete(void);
void ENGINE_load_builtin_engines(void);

in crypto/apr_crypto_openssl.c lines 141, 142, 208, 209, 326, 330, 331.

Best regards,

Rainer

Re: [VOTE] release apr-util-1.6.2-rc2 as apr-util 1.6.2

[Rainer Jung]

Am 23.01.23 um 19:57 schrieb Eric Covener:

1.6.2-rc2 is here:

https://apr.apache.org/dev/dist/

For the release of apr-util-1.6.2
   [ X]  +1 looks great!
   [  ]  -1 something is broken

I will let the vote run through mid-week and then try to finalize APR
and APU on Thursday if I can, else early the week after.


+1 to release and thanks a bunch for RMing.

I haven't checked the ongoing baxckport discussion, so I am neutral on this.

Built and tested on SLES 11, 12 and 15, RHEL 6, 7 and 8 plus Solaris 10 
Sparc. Tested also in combination with apr-util 1.6.2-rc2 and unit tests 
with httpd 2.4.55-rc1.


crypto and httpd were compiled against OpenSSL 3.1.0beta1, 3.0.7, 
1.1.1s, 1.0.2u and tested with clients using OpenSSL 3.1.0beta1, 3.0.0, 
1.1.1g (with some patches), 1.0.2u and 1.1.0l.


No regressions, some - may expected - TLS failures in httpd when testing 
client 3.1.0beta1 against server 1.0.2u.


Best regards,

Rainer

Minor nag: APR build warnings on Solaris

[Rainer Jung]

During build and test for the 1.7.1-rc2 release vote I noticed the 
following warnings, but only for Solaris 10 Sparc. None is new, they 
were already present at least for 1.7.0:


locks/unix/proc_mutex.c:979:49: warning: 
'mutex_proc_pthread_cond_methods' defined but not used 
[-Wunused-const-variable=]


and in the tests:

test/testatomic.c:331:56: warning: format '%lu' expects argument of type 
'long unsigned int', but argument 3 has type 'apr_uint64_t' {aka 'long 
long unsigned int'} [-Wformat=]
test/testatomic.c:345:57: warning: format '%lu' expects argument of type 
'long unsigned int', but argument 3 has type 'apr_uint64_t' {aka 'long 
long unsigned int'} [-Wformat=]


Best regards,

Rainer

Re: VOTE: Release apr-1.7.1-rc2 as 1.7.1

[Rainer Jung]

Am 20.01.23 um 01:44 schrieb Eric Covener:

1.7.1-rc1 is here:

https://apr.apache.org/dev/dist/

For the release of apr-1.7.1
   [ X]  +1 looks great!
   [  ]  -1 something is broken

I will let the vote run through mid-week.


+1 to release and thanks a bunch for RMing.

Build and tested on SLES 11, 12 and 15, RHEL 6, 7 and 8 plus Solaris 10 
Sparc. Tested also in combination with apr-util 1.6.2-rc2 and unit tests 
with httpd 2.4.55-rc1.


More details about the tested setups in my forthcomoing apr-util vote.

Best regards,

Rainer

Re: svn commit: r1863234 - in /apr/apr/branches/1.7.x: ./ include/apr_pools.h memory/unix/apr_pools.c

[Rainer Jung]

Thanks!

Am 29.06.2022 um 17:01 schrieb Yann Ylavic:

On Wed, Jun 29, 2022 at 4:41 PM William A Rowe Jr  wrote:


This is the first commit which appeared to break APR in such a way
that code compiled
against apr 1.7.1 would be unable to run against apr 1.7.0 binaries.


I think it has been reverted already by http://svn.apache.org/r1902022

Regards;
Yann.

Re: svn commit: r1863234 - in /apr/apr/branches/1.7.x: ./ include/apr_pools.h memory/unix/apr_pools.c

[Rainer Jung]

Thanks, I will revert.

Am 29.06.2022 um 16:40 schrieb William A Rowe Jr:

This is the first commit which appeared to break APR in such a way
that code compiled
against apr 1.7.1 would be unable to run against apr 1.7.0 binaries.

Thoughts, rjung? Are you willing to revert? This would persist on apr
1.8.x for those
interested in this new feature.

On Wed, Jul 17, 2019 at 2:04 PM  wrote:


Author: rjung
Date: Wed Jul 17 19:04:43 2019
New Revision: 1863234

URL: http://svn.apache.org/viewvc?rev=1863234&view=rev
Log:
Add empty stubs for APR pool functions, that are
only needed when APR_POOL_DEBUG is defined,
but can be aor should be called from app code.

Providing stubs allows the app code to stay the
same when running with or without debugging APR
lib (no need for app recompilation).

Backport of r1863217 from trunk.

Modified:
 apr/apr/branches/1.7.x/   (props changed)
 apr/apr/branches/1.7.x/include/apr_pools.h
 apr/apr/branches/1.7.x/memory/unix/apr_pools.c

Propchange: apr/apr/branches/1.7.x/
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Wed Jul 17 19:04:43 2019
@@ -1,4 +1,4 @@
  /apr/apr/branches/1.4.x:1003369,1101301
-/apr/apr/trunk:733052,739635,741862,741866-741867,741869,741871,745763-745764,746310,747990,748080,748361,748371,748565,74,748902,748988,749810,760443,767895,775683,782838,783398,783958,784633,784773,788588,789050,793192-793193,794118,794485,795267,799497,800627,809745,809854,810472,811455,813063,821306,829490,831641,832904,835607,888669,892028,892159,892435,892909,896382,896653,899905,901088,902077,902090,908427,910419,910597,917819,917837-917838,923311,923320,925965,929796,930508,931973,932585,951771,960665,960671,979891,983618,989450,990435,1003338,100,107,1055657,1072165,1078845,1081462,1081495,1082177,1083038,1083242,1084662,1086695,1088023,1089031,1089129,1089438,1099348,1103310,1183683,1183685-1183686,1183688,1183693,1183698,1213382,1235047,1236970,1237078,1237507,1240472,1340286,1340288,1340470,1341193,1341196,1343233,1343243,1367050,1368819,1370494,1372018,1372022,1372093,1372849,1376957,1384764,1389077,1400200,1402868,1405985,1406690,1420106,1420109,1425356,142
  
8809,1438940,1438957-1438959,1442903,1449568,1456418,1459994,1460179-1460180,1460184,1460241,1460399,1460405,1462738,1462813,1470186,1470348,1475509,1478905,1480067,1481186,1481262,1481265,1484271,1487796,1489517,1496407,1502804,1510354,1516261,1523384,1523479,1523484,1523505,1523521,1523604,1523613,1523615,1523844-1523845,1523853,1524014,1524031,1528797,1528809,1529488,1529495,1529515,1529521,1529668,1530786,1530800,1530988,1531554,1531768,1531884,1532022,1533104,1533111,1533979,1534882,1535027,1535157,1536744,1538171,1539374,1539389,1539455,1539603,1541054,1541061,1541486,1541655,1541666,1541744,1542601,1542779,1543033,1543056,1548575,1550907,1551650,1551659,1558905,1559382,1559873,1559975,1561040,1561260,1561265,1561321,1561347,1561356,1561361,1561394,1561555,1571894,1575509,1578420,1587045,1587063,1587543,1587545,1588878,1588937,1589982,1593611,1593614-1593615,1593680,1594684,1594708,1595549,1597797,1597803,1604590,1604596,1604598,1605104,1610854,1611023,1611107,160,167,
  
1611120,1611125,1611184,1611193,1611466,1611515,1611517,1625173,1626564,1634615,1642159,1648830,1664406,1664447,1664451,1664471,1664769-1664770,1664775,1664904,1664911,1664958,1666341,1666411,1666458,111,1667420-1667421,1667423,1667900-1667901,1667903,1667914-1667916,1667962,1669077,1671292,1671329,1671356,1671386,1671389,1671513-1671514,1671957,1672354,1672366,1672495,1672575,1674566,1675644,1675656,1675668,1675967,1675970,1675982,1676013,1683521,1685929,1696140,1696767,1722547,1722557,1726928,1727020,1727160,1727175,1727199,1728957,1732582,1733451,1733594,1733694,1733706,1733708,1733775,1734816,1736552,1738791,1738925,1750374,1755709,1755740,1755746,1755758,1755954,1761279,1762326,1774712,1774973,1775069,1776994,1776998,1788334,1788337,1788929,1789947,1789998,1790045,1790200,1790296,1790302-1790304,1790330-1790331,1790436,1790439,1790444,1790446,1790488,1790521,1790523,1790569,1790632,1791598,1791718,1791728,1792620-1792622,1792625,1792961,1792963,1794266,1797415,1798105,18053
  
80,1808039,1808836,1808910,1809649,1810452,1813286,1813330,1814239-1814240,1814326,1814329,1814331,1816527,1816628,1817485,1819857-1819858,1819860-1819861,1819934-1819935,1820080,1820755,1822357,1827534,1832203,1832691,1832985,1834253,1834494,1834541,1836235,1839068,1839493,1839622,1839769,1840372,1841078,1846806,1850087,1850095,1851541-1851542,1854123,1855049,1855347,1855443-1855444,1855839-1855840,1855855,1855864,1855867,1855877,1855949,1856022,1856042-1856043,1856046,1856050,1856063,1856089,1856096,1856178,1856192,1856196,1856756,1858596,1863205
+/apr/apr/trunk:733052,739635,741862,741866-741867,741869,741871,745763-745764,746310,747990,748080,748361,748371,748565,74,748902,748988,749810,760443,767895,775683,782838,783398,783958,784633,784773,7

Re: apr-util-1.7.0 initial release, or 1.6.x incremental release (or both)?

[Rainer Jung]

AFAIK if you want to build including crypto support, 1.7.x needs OpenSSL 
1.0.2+. So if you need to stick to eg. 0.9.8, you are limited to 1.6.x. 
As we know, Apache httpd 2.4.x eg. still supports 0.9.8. So anyone 
building a recent 2.4.x using the totally outdated 0.9.8 must use APU 1.6.x.


Concerning more 1.6.x updates: my understandig of the versioning was, 
that the middle digit provides API stability, ie. no new APIs are 
included in 1.N.x once the first patch release 1.N.0 was released. Based 
on that I would suggest to update 1.6.x very conservativly, mostly real 
bug fixes, no features. I would for instance favor not to include libxml 
support in 1.6.x.


Deciding about having a 1.6.x release could depend on how important we 
think those committed fixes are.


Just my 2c.

Best regards,

Rainer

Am 20.09.2021 um 11:35 schrieb Yann Ylavic:

On Fri, Sep 17, 2021 at 9:53 PM William A Rowe Jr  wrote:


We have much activity on apr-util-1.7.0 but no release actions for
some time now.
There are very minor changes to 1.6.x branch.

Do we want to release 1.7.0, or only a 1.6.x bump?
Or do we want to do both? Are we ready and are the API changes considered?

All observations and feedback on apr-util releases encouraged...


I tried to align apr-util 1.6.x and 1.7.x w.r.t. "bug fixes" in case
we want to release both, but I think that it's not necessarily worth
it.

Looking at the diffs between the two here is what I found:

* Build with Mariadb not in 1.6.x:
- http://svn.apache.org/viewvc?rev=1872061&view=rev

* Build with libxml2/expat work in 1.7.x only (I think 1.6.x would
want it too?):
- http://svn.apache.org/viewvc?rev=1854603&view=rev
- http://svn.apache.org/viewvc?rev=1880287&view=rev

(Also on the xml side still, do we want Windows' xmllite work in 1.7.x? Ivan?)

* The crypto PRNG and Digests are new to 1.7.x (not 1.6.x material per
our compat rules).
   I'm not sure that 1.6.x is fine with openssl-1.1(.1)+,
OPENSSL_init_crypto() is in 1.7.x only (we probably want these changes
in both).

* apr_dbm_driver_t and apr_dbm_get_driver() are new to 1.7.x (not
1.6.x material per our compat rules).

* apu_dso_init/load() fixes w.r.t. global pool scope in 1.7.x only (I
think we'd want to align here too).

So if we want to release both, should we look at the above points that
can align?


Regards;
Yann.

Re: APR trunk testcrypto crash with OpenSSL 1.0.2

[Rainer Jung]

I did not observe this crash with apr-util 1.7.x nor with 1.6.x although 
both HEADS were tested as well. I hope this was not just a limitation of 
my test setup, but at least I am not aware of such a limitation.


Regards,

Rainer

Am 17.09.2021 um 19:49 schrieb William A Rowe Jr:

Did we determine this does not affect apr-util-1.7.x nor
apr-util-1.6.x branches?

Re: APR trunk testcrypto crash with OpenSSL 1.0.2

[Rainer Jung]

Confirmed: crash no longer occurs with current APR trunk.

Thanks a bunch!

Rainer

Am 12.09.2021 um 13:56 schrieb Yann Ylavic:

Hi Rainer,

On Sat, Sep 11, 2021 at 10:15 PM Rainer Jung  wrote:


I get crashes with APR trunk testcrypto when running with OpenSSL 1.0.2
(at least on Linux).

Here's an example:

*** glibc detected *** ./testall: double free or corruption (!prev):
0x021e2d40 ***


Thanks for the report, should be fixed (hopefully) in r1893270.

Regards;
Yann.

APR trunk testcrypto crash with OpenSSL 1.0.2

[Rainer Jung]

Hi there,

I get crashes with APR trunk testcrypto when running with OpenSSL 1.0.2 
(at least on Linux).


Here's an example:

*** glibc detected *** ./testall: double free or corruption (!prev): 
0x021e2d40 ***



#0  0x003906b8cee8 in main_arena () from /lib64/libc.so.6
No symbol table info available.
#1  0x7762f027 in EVP_CIPHER_CTX_cleanup (c=0x6f4770) at 
evp_enc.c:593

No locals.
#2  0x7762f22e in EVP_CIPHER_CTX_free (ctx=0x6f4770) at 
evp_enc.c:584

No locals.
#3  0x77953021 in cprng_stream_ctx_free (sctx=0x6f7e30) at 
/path/to/apr/trunk/r1893236/crypto/apr_crypto_openssl.c:1544

No locals.
#4  0x77da89f6 in cprng_cleanup (arg=0x6f7540) at 
/path/to/apr/trunk/r1893236/crypto/apr_crypto_prng.c:253

cprng = 0x6f7540
#5  0x77da8e8c in apr_crypto_prng_create (pcprng=0x77ffb1e8, 
crypto=, cipher=, 
bufsize=,
flags=1, seed=, pool=0x6f7458) at 
/path/to/apr/trunk/r1893236/crypto/apr_crypto_prng.c:388

rv = 
cprng = 0x6f7540
#6  0x77da9268 in apr_crypto_prng_init (pool=0x6f7458, 
crypto=0x0, cipher=APR_CRYPTO_CIPHER_CHACHA20, bufsize=0, seed=optimized out>,
flags=) at 
/path/to/apr/trunk/r1893236/crypto/apr_crypto_prng.c:174

rv = 0
#7  0x0042fefa in test_crypto_prng (tc=0x7fffe340, 
cipher=APR_CRYPTO_CIPHER_CHACHA20,
test_PRNG_kat0=0x44e320 
"\260\375\024\377\226\240\275\241T\303)\b,\234e3\273L\224s\277]\336\023\217\202ɬUS\331X\257\275\255(E\271<۲\376dc\322\376\026*\332\340\366\346v\360IB\030\365\316\005\226\347\237\\U\032\252\233\244o\252\325(\366v=ޓ\300?\243\261!\262\377\300S:i^\325n\217\332\005\211\242\355\353\372\324\256\323\065|z\255\255\223(\002{\270y\265WG\227\241\267=\316|П$Q\001y\004*3\372A\032\067\227:\354\240\374\336k+\026\244_\241*\343\365L\204(\203\353`\316D\351\234L\242np\314&h\370\231Z\241\237ޙ\271\200\v\266\203\024\235r\223\364\321I\363\360\236I\200v\204\001\036y\236pp"...) 
at /path/to/apr/trunk/r1893236/test/testcrypto.c:2457
randbytes = 
"p\002C\000\000\000\000\000\210\357\204\006\071\000\000\000\030\000\000\000\060\000\000\000\060\343\377\377\377\177\000\000p\342\377\377\377\177\000\000\276\367\272\311!\002\207w\256\277\312)\340\224\202I\213\375_\004\255\355\061M/\000\000\000\000\000\000\000\001", 
'\000' , 
"\005\000\000\000\000\000\000\000ACEGIKMOQSUWY[]_acegikmo\243\023\207\006\071\000\000"
seed = 
"\000\000\000\000\000\000\000\000\200\267\270\006\071\000\000\000\002\000\000\000\000\000\000\000\000\320\330\367\377\177\000"

cprng = 0x0
pool = 0x6f7458
rv = 
i = 
flags = 2
#8  0x0040c9c9 in abts_run_test (ts=, 
f=0x430270 , value=0x0)

at /path/to/apr/trunk/r1893236/test/abts.c:192
tc = {failed = 0, skipped = 0, suite = 0x6cc1c0}
ss = 0x6cc1c0
#9  0x0042f783 in testcrypto (suite=0x6c91a0) at 
/path/to/apr/trunk/r1893236/test/testcrypto.c:2772


Best regards,

Rainer

APR trunk xml interface: libxml2 impl not conforming

[Rainer Jung]

Hi there,

"make check" für current APR trunk fails in testxml when used with libxml2:

  testxml :  Line 298: expected <20014>, but saw <73>


73 probably cpomes from here:

  include/libxml2/libxml/xmlerror.h:XML_ERR_GT_REQUIRED, /* 73 */

and the 20014 is the expected APR_EGENERAL.

The expat implementation contains in xml/apr_xml_expat.c:

 45 static apr_status_t do_parse(apr_xml_parser *parser,
 46  const char *data, apr_size_t len,
 47  int is_final)
 48 {
...
 53 int rv = XML_Parse(parser->xp, data, (int)len, is_final);
 54
 55 if (rv == 0) {
 56 parser->error = APR_XML_ERROR_EXPAT;
...
 59 }
 60 }
 61
 62 /* ### better error code? */
 63 return parser->error ? APR_EGENERAL : APR_SUCCESS;
 64 }
 65
 66
 67 static XMLParserImpl xml_parser_expat = {
 68 do_parse,
 69 cleanup_parser
 70 };

For libxml2 we have in xml/apr_xml_libxml2.c:

 39 static int libxml2_parse(apr_xml_parser* parser, const char* data,
 40  apr_size_t sz, int final)
 41 {
 42 parser->xp_err = xmlParseChunk(parser->xp, data, sz, final);
 43 if (parser->xp_err != 0) {
 44 xmlErrorPtr errptr = xmlCtxtGetLastError(parser->xp);
 45 parser->xp_msg = errptr->message;
 46 /* this misnomer is used as a test for (any) parser error. */
 47 parser->error = APR_XML_ERROR_EXPAT;
 48 }
 49 return parser->xp_err;
 50 }
 51 static XMLParserImpl xml_parser_libxml2 = {
 52 libxml2_parse,
 53 cleanup_parser
 54 };
 55

So the libxml2 implementation returns native libxml2 error codes, the 
expat impl returns APR error codes. Since in the xml interface the 
declaration is


apr_status_t (*Parse)(apr_xml_parser*, const char*, apr_size_t, int);

I think the libxml2 impl needs fixing.

Best regards,

Rainer

Re: APR 1.7.1 release?

[Rainer Jung]

Am 03.09.2021 um 03:44 schrieb William A Rowe Jr:

I'm willing to RM APR and APR-util 1.7 releases.


That would be great.


Would propose we set a date out 2 weeks, anything lingering needs
to be finalized with the usual oversight no later than the 8th, and
we tag on the 14th, announce on the 15th when the mirrors have
caught up. That gives enough days for committers to review the
last changes to these release branches.


Sounds like a plan, though I can't judge on which important things need 
to be fixed. Hopefully nothing.



But I'd be happier co-RM'ing this with a newer committer/PMC
participant who wants to learn the ropes. Any volunteers?
Other thoughts or observations?


Regards,

Rainer

APR 1.7.1 release?

[Rainer Jung]

Hi there,

any chance we find an RM for a APR 1.7.1 release? At least there was the 
fix for CVE-2021-35940 and CHANGES contains 15 more items (many of them 
platform specific or build improvements). Last release 1.7.0 was in 
April 2019.


For APR-util I don't know the current state and release needs for the 
1.6.x and 1.7.x branches. Last 1.6.x release was in October 2017, 1.7.x 
has never been released. CHANGES for 1.6.x only contains one 
apr_dbm_gdbm fix plus a minor libtool use improvement.


Apache httpd is planing to start a release cycle soon and it would be 
nice to have a clean APR 1.7.1 and maybe APR-util also.


Thanks and regards,

Rainer

Re: svn commit: r1884109 - in /apr/apr-util/branches/1.6.x: ./ misc/apr_thread_pool.c

[Rainer Jung]

Confirmed again, patch works for me. Thanks!

Am 25.04.2021 um 16:23 schrieb Rainer Jung:
Thanks a bunch Yann, looks very promising. Testing is ongoing and I will 
report back later today, but I did not yet observe any hangs with the 
patch although on the specific platform I am testing now (SLES 12) they 
happened pretty frequent without the patch.


Best regards,

Rainer

Am 25.04.2021 um 13:36 schrieb Yann Ylavic:
On Fri, Apr 23, 2021 at 11:26 AM Rainer Jung  
wrote:


When compiling and running make check for latest head of APU 1.6.x in
preparation of httpd release testing, I observe very frequent hangs
during testqueue and testpass.

It seems the hang happens only for APU 1.6.x head, but not for APU 
1.6.1.


The hangs seem not to depend on the exact APR version. I observed them
with 1.6.5, 1.7.0, 1.6.x head and 1.7.x head.

I have not yet tested APU 1.7.x head.

I can see them on RHEL 6, SLES 11, 12 and 15 and Solaris. It seems they
do not happen on RHEL 7 and 8.

Here are some stacks:


Thanks Rainer for the details.

Does the attached patch (for 1.6.x) fix the hang?

Regards;
Yann.

Re: svn commit: r1884109 - in /apr/apr-util/branches/1.6.x: ./ misc/apr_thread_pool.c

[Rainer Jung]

Thanks a bunch Yann, looks very promising. Testing is ongoing and I will 
report back later today, but I did not yet observe any hangs with the 
patch although on the specific platform I am testing now (SLES 12) they 
happened pretty frequent without the patch.


Best regards,

Rainer

Am 25.04.2021 um 13:36 schrieb Yann Ylavic:

On Fri, Apr 23, 2021 at 11:26 AM Rainer Jung  wrote:


When compiling and running make check for latest head of APU 1.6.x in
preparation of httpd release testing, I observe very frequent hangs
during testqueue and testpass.

It seems the hang happens only for APU 1.6.x head, but not for APU 1.6.1.

The hangs seem not to depend on the exact APR version. I observed them
with 1.6.5, 1.7.0, 1.6.x head and 1.7.x head.

I have not yet tested APU 1.7.x head.

I can see them on RHEL 6, SLES 11, 12 and 15 and Solaris. It seems they
do not happen on RHEL 7 and 8.

Here are some stacks:


Thanks Rainer for the details.

Does the attached patch (for 1.6.x) fix the hang?

Regards;
Yann.

Re: svn commit: r1884109 - in /apr/apr-util/branches/1.6.x: ./ misc/apr_thread_pool.c

[Rainer Jung]

When compiling and running make check for latest head of APU 1.6.x in 
preparation of httpd release testing, I observe very frequent hangs 
during testqueue and testpass.


It seems the hang happens only for APU 1.6.x head, but not for APU 1.6.1.

The hangs seem not to depend on the exact APR version. I observed them 
with 1.6.5, 1.7.0, 1.6.x head and 1.7.x head.


I have not yet tested APU 1.7.x head.

I can see them on RHEL 6, SLES 11, 12 and 15 and Solaris. It seems they 
do not happen on RHEL 7 and 8.


Here are some stacks:

RHEL 6

#0  0x003906c0b44c in pthread_cond_wait@@GLIBC_2.3.2 () from 
/lib64/libpthread.so.0
#1  0x7fe39c453cb5 in apr_thread_pool_tasks_cancel () from 
/shared/build/dev/httpd/build/apr-util/1.6.x/r1889104/rhel6.x86_64-modular_enable-apr_1.6.x_r1876940-dso_enable-expat_2.3.0-1-ldap_2.4.58-300-1-openssl_3.0.0alpha15-1-shared-sqlite_3.35.5-1-bdb_6.1.19-1/.libs/libaprutil-1.so.0
#2  0x7fe39c453d45 in thread_pool_cleanup () from 
/shared/build/dev/httpd/build/apr-util/1.6.x/r1889104/rhel6.x86_64-modular_enable-apr_1.6.x_r1876940-dso_enable-expat_2.3.0-1-ldap_2.4.58-300-1-openssl_3.0.0alpha15-1-shared-sqlite_3.35.5-1-bdb_6.1.19-1/.libs/libaprutil-1.so.0
#3  0x7fe39bff6c6e in apr_pool_destroy () from 
/shared/build/dev/httpd/install/apr/1.6.x/r1876940/rhel6.x86_64-dso_enable/lib/libapr-1.so.0
#4  0x7fe39c45492c in apr_thread_pool_destroy () from 
/shared/build/dev/httpd/build/apr-util/1.6.x/r1889104/rhel6.x86_64-modular_enable-apr_1.6.x_r1876940-dso_enable-expat_2.3.0-1-ldap_2.4.58-300-1-openssl_3.0.0alpha15-1-shared-sqlite_3.35.5-1-bdb_6.1.19-1/.libs/libaprutil-1.so.0

#5  0x0040853c in test_threadsafe ()
#6  0x0040610c in abts_run_test ()
#7  0x004081e2 in testpass ()
#8  0x00405675 in main ()

SLES 11

#1  0x2b1e34104315 in apr_thread_pool_tasks_cancel () from 
/path/to/httpd/build/apr-util/1.6.x/r1889104/sles11.x86_64-modular_enable-apr_1.6.5-dso_enable-expat_2.3.0-1-ldap_2.4.58-111-1-openssl_1.1.1k-1-shared-sqlite_3.35.5-1-bdb_6.1.19-1/.libs/libaprutil-1.so.0
#2  0x2b1e34104395 in thread_pool_cleanup () from 
/path/to/httpd/build/apr-util/1.6.x/r1889104/sles11.x86_64-modular_enable-apr_1.6.5-dso_enable-expat_2.3.0-1-ldap_2.4.58-111-1-openssl_1.1.1k-1-shared-sqlite_3.35.5-1-bdb_6.1.19-1/.libs/libaprutil-1.so.0
#3  0x2b1e3455dcff in apr_pool_destroy () from 
/path/to/httpd/install/apr/1.6.x/1.6.5/sles11.x86_64-dso_enable/lib/libapr-1.so.0
#4  0x2b1e34104fdc in apr_thread_pool_destroy () from 
/shared/build/dev/httpd/build/apr-util/1.6.x/r1889104/sles11.x86_64-modular_enable-apr_1.6.5-dso_enable-expat_2.3.0-1-ldap_2.4.58-111-1-openssl_1.1.1k-1-shared-sqlite_3.35.5-1-bdb_6.1.19-1/.libs/libaprutil-1.so.0

#5  0x0040b7e0 in test_queue_producer_consumer ()
#6  0x004067ed in abts_run_test ()
#7  0x0040b611 in testqueue ()
#8  0x00405d6d in main ()

SLES 12

#0  0x7f39ed06c05f in pthread_cond_wait@@GLIBC_2.3.2 () from 
/lib64/libpthread.so.0
#1  0x7f39ed8e6c2d in apr_thread_cond_wait (cond=, 
mutex=) at 
/shared/build/dev/httpd/sources/apr/1.7.x/r1889104/locks/unix/thread_cond.c:68
#2  0x7f39edd49cd1 in wait_on_busy_threads (owner=0x0, me=0x20c5630) 
at 
/shared/build/dev/httpd/sources/apr-util/1.6.x/r1889104/misc/apr_thread_pool.c:768
#3  apr_thread_pool_tasks_cancel (me=me@entry=0x20c5630, 
owner=owner@entry=0x0) at 
/shared/build/dev/httpd/sources/apr-util/1.6.x/r1889104/misc/apr_thread_pool.c:793
#4  0x7f39edd49fe5 in thread_pool_cleanup (me=0x20c5630) at 
/shared/build/dev/httpd/sources/apr-util/1.6.x/r1889104/misc/apr_thread_pool.c:397
#5  0x7f39ed8e7f9e in run_cleanups (cref=0x20dd3d8) at 
/shared/build/dev/httpd/sources/apr/1.7.x/r1889104/memory/unix/apr_pools.c:2705
#6  apr_pool_destroy (pool=0x20dd368) at 
/shared/build/dev/httpd/sources/apr/1.7.x/r1889104/memory/unix/apr_pools.c:978
#7  0x7f39edd4995c in apr_thread_pool_destroy (me=) 
at 
/shared/build/dev/httpd/sources/apr-util/1.6.x/r1889104/misc/apr_thread_pool.c:457
#8  0x0040b0e1 in test_queue_producer_consumer (tc=0x21de5d0, 
data=) at 
/shared/build/dev/httpd/sources/apr-util/1.6.x/r1889104/test/testqueue.c:120
#9  0x004060ee in abts_run_test (ts=, f=0x40afe0 
, value=0x0) at 
/shared/build/dev/httpd/sources/apr-util/1.6.x/r1889104/test/abts.c:172
#10 0x0040b231 in testqueue (suite=0x20c7160) at 
/shared/build/dev/httpd/sources/apr-util/1.6.x/r1889104/test/testqueue.c:131
#11 0x00405a3d in main (argc=, argv=out>) at 
/shared/build/dev/httpd/sources/apr-util/1.6.x/r1889104/test/abts.c:416


SLES 15

#0  0x7f41a1a2b89d in pthread_cond_wait@@GLIBC_2.3.2 () from 
/lib64/libpthread.so.0
#1  0x7f41a22a576d in apr_thread_cond_wait (cond=, 
mutex=) at 
/path/to/sources/apr/1.6.x/r1876940/locks/unix/thread_cond.c:68
#2  0x7f41a270d3d9 in wait_on_busy_threads (owner=0x0, me=0x1164870) 
at /path/to/sources/apr-util/1.6.x/r1889104/misc/apr_thread_pool.c:768
#3  apr_thr

Re: svn commit: r1836445 [1/3] - in /apr/apr-util/branches/1.7.x: ./ crypto/ include/ include/private/ test/

[Rainer Jung]

Coming back to this 6 months later: the situation is still the same.
Futhermore the same commit broke compat with OpenSSL 0.9.8 without 
documenting the increased version requirements.


It would be good to fix this.

Regards,

Rainer

Am 07.08.2019 um 00:55 schrieb Rainer Jung:
Solaris(10) provides NSS support but it is somewhat outdated. Especially 
it does not know about SEC_OID_SHA224, so compilation (linking) fails.


So currently APR trunk and APR-UTIL 1.7.x no longer compile on Solaris 
(1.7.x has more problems outside of crypto).


We might need to detect SEC_OID_SHA224 during configure and find a way 
to cope with its nonexistence in the code.


I tried compilation with the dirty workaroung:

--- crypto/apr_crypto_nss.c    2019-08-04 12:00:19.841885000 +
+++ crypto/apr_crypto_nss.c    2019-08-07 00:28:16.516409000 +
@@ -736,7 +736,8 @@
  key->hashAlg = SEC_OID_SHA1;
  break;
  case APR_CRYPTO_DIGEST_SHA224:
-    key->hashAlg = SEC_OID_SHA224;
+    return APR_ENODIGEST;
+    //key->hashAlg = SEC_OID_SHA224;
  break;
  case APR_CRYPTO_DIGEST_SHA256:
  key->hashAlg = SEC_OID_SHA256;

and that compiles on Solaris 10 Sparc. And "make check" is fine, except 
of course in testcrypto the specific tests for DIGEST SHA224 nss.


Regards,

Rainer

Re: svn commit: r1836445 [1/3] - in /apr/apr-util/branches/1.7.x: ./ crypto/ include/ include/private/ test/

[Rainer Jung]

Solaris(10) provides NSS support but it is somewhat outdated. Especially 
it does not know about SEC_OID_SHA224, so compilation (linking) fails.


So currently APR trunk and APR-UTIL 1.7.x no longer compile on Solaris 
(1.7.x has more problems outside of crypto).


We might need to detect SEC_OID_SHA224 during configure and find a way 
to cope with its nonexistence in the code.


I tried compilation with the dirty workaroung:

--- crypto/apr_crypto_nss.c 2019-08-04 12:00:19.841885000 +
+++ crypto/apr_crypto_nss.c 2019-08-07 00:28:16.516409000 +
@@ -736,7 +736,8 @@
 key->hashAlg = SEC_OID_SHA1;
 break;
 case APR_CRYPTO_DIGEST_SHA224:
-key->hashAlg = SEC_OID_SHA224;
+return APR_ENODIGEST;
+//key->hashAlg = SEC_OID_SHA224;
 break;
 case APR_CRYPTO_DIGEST_SHA256:
 key->hashAlg = SEC_OID_SHA256;

and that compiles on Solaris 10 Sparc. And "make check" is fine, except 
of course in testcrypto the specific tests for DIGEST SHA224 nss.


Regards,

Rainer

Re: svn commit: r1854603 - in /apr/apr-util/branches/1.7.x: build/apu-conf.m4 build/xml.m4 configure.in include/apu.h.in include/apu.hw include/apu.hwc xml/NWGNUmakefile xml/apr_xml.c xml/apr_xml_expa

[Rainer Jung]

Just a reminder, that APR-UTIL 1.7.x still doesn't compile due to the 
wrong handling of INCLUDES.


Am 30.03.2019 um 14:31 schrieb Rainer Jung:

Hi Nick,

Am 27.03.2019 um 23:42 schrieb Nick Kew:




On 27 Mar 2019, at 17:55, Rainer Jung  wrote:

This breaks 1.7 apu compilation using non-system expat for me. The 
expat detection sets INCLUDES but during compilation INCLUDES isn't 
used anywhere, so expat.h can't be found.


OK, I need to return to that.  Thanks for the reminder!

Does trunk compile for you in the same circumstances?
This is supposed to be a backport!


Trunk is APR which adds INCLUDES to EXTRA_INCLUDES and uses those during 
build. APU doesn't use EXTRA_INCLUDES anywhere but uses the 
APRUTIL_PRIV_INCLUDES and APRUTIL_INCLUDES variables, none of which the 
expat flags gets added to. dbd seems to use APRUTIL_PRIV_INCLUDES, all 
other dependencies seem to use APRUTIL_INCLUDES ...


Regards,

Rainer

Re: Resolutions to APR_POOL_DEBUGging headaches?

[Rainer Jung]

Am 24.07.2019 um 01:07 schrieb William A Rowe Jr:
I'm sorry I haven't had the summertime cycles to dig into this, but am 
wondering on the cusp of an httpd 2.4.40, we've learned a bunch about 
possible apr pool and especially debug traps that meta


What code remains to be considered for inclusion in an apr 1.7.1?


I have added all trunk pool debug changes to 1.7.x that I found missing. 
IMHO what's left changes API and would be for 1.8, see below.



What is apr 1.8 waiting for, if we think we can slightly twist things?


From the point of view of pool debugging (the only thing I looked at):

r1817892 adding apr_pool_get_tag (only indirectly a pool debug feature)
r1460182 adding apr_pool_owner_set
The parts of r1863217 that are again about apr_pool_owner_set

Do we agree you can't jump between APR_POOL_DEBUG and retain ABI to the 
non-debug build?


ABI as defined "binary compatible" in the APR docs:

=== SNIP ===
We define "binary compatible" to mean that a compiled application 
can be linked (possibly dynamically) against the library and continue to 
function properly.


Similar to source compatibility, an application that has been 
compiled against a particular version will continue to be linkable 
against later versions (unless the major number changes). It is possible 
that an application will not be able to successfully link against a 
previous minor version.

=== SNIP ===

IMHO it should be possible to replace a normal build of the apr library 
by a APR_POOL_DEBUG build of it and run unchanged apps using it, so they 
should be "binary compatible". This should be easy due to the fact, that 
the relevant structs, that change their behavior for debug builds, 
should be opaque, so access only happens via the API and not directly on 
the structs.


Furthermore apps should have no need to use different code depending on 
the APR_POOL_DEBUG define. If they need to instruct the debug library 
about specific decisions, like using apr_pool_join(), that code should 
be usable unchanged with a non-debug apr lib (by providing the calls as 
no-ops in the library. IMHO this is currently the case for trunk, 1.7.x 
and 1.6.x (only HEAD, after my changes last week).


Have we eliminated the cause for APR 1.x folks to ship a release based 
on APR_POOL_DEBUG?


For 1.7.x and 1.6.x (HEADS) I do think so, more eyes welcome.

What proposals are we still working to commit to apr trunk, for deep 
inspection within snapshot/dev tooling builds?


None here.


Which other questions did I overlook?

Maybe a whiteboard (wiki page) for all the proposals under 
consideration, or assemble these under STATUS on trunk?


Regards,

Rainer

Re: crash during httpd cleanup when using APR debug library (APR_POOL_DEBUG)

[Rainer Jung]

Thanks Rüdiger. I hadn't expected it to be fixed in trunk long ago. I 
see now, that there are more useful pool debug backports sitting in 
trunk. Will look at it soon.


Regards,

Rainer

Am 17.07.2019 um 12:09 schrieb Ruediger Pluem:



On 07/17/2019 11:43 AM, Rainer Jung wrote:

Am 17.07.2019 um 10:03 schrieb Ruediger Pluem:



On 07/16/2019 11:28 PM, Rainer Jung wrote:

cross-posted to APR+HTTPD

Crahs happens in #2  0x7faf4c154945 in raise () from /lib64/libc.so.6
#3  0x7faf4c155f21 in abort () from /lib64/libc.so.6
#4  0x7faf4c14d810 in __assert_fail () from /lib64/libc.so.6
#5  0x7faf4c694219 in __pthread_tpp_change_priority () from 
/lib64/libpthread.so.0
#6  0x7faf4c68cd76 in __pthread_mutex_lock_full () from 
/lib64/libpthread.so.0
#7  0x7faf4cd07c29 in apr_thread_mutex_lock (mutex=0x2261fe0) at 
locks/unix/thread_mutex.c:108
#8  0x7faf4cd08603 in apr_pool_walk_tree (pool=0x225a710, fn=0x7faf4cd07fc0 
, data=0x7faf45777c90)
at memory/unix/apr_pools.c:1515
#9  0x7faf4cd08630 in apr_pool_walk_tree (pool=0x6a3ce0, fn=0x7faf4cd07fc0 
, data=0x7faf45777c90) at
memory/unix/apr_pools.c:1521
#10 0x7faf4cd08630 in apr_pool_walk_tree (pool=0x6a3770, fn=0x7faf4cd07fc0 
, data=0x7faf45777c90) at
memory/unix/apr_pools.c:1521
#11 0x7faf4cd08630 in apr_pool_walk_tree (pool=0x6a3110, fn=0x7faf4cd07fc0 
, data=0x7faf45777c90) at
memory/unix/apr_pools.c:1521
#12 0x7faf4cd086df in apr_pool_num_bytes (pool=0x6d81, recurse=) at
memory/unix/apr_pools.c:2304
#13 0x7faf4cd0898f in apr_pool_log_event (pool=0x225a710, event=0x7faf4cd16e74 
"PCALLOC", file_line=0x7faf4cd16d78
"locks/unix/thread_mutex.c:50", deref=-1)
  at memory/unix/apr_pools.c:1543
#14 0x7faf4cd098b8 in apr_pcalloc_debug (pool=0x225a710, size=64, 
file_line=0x7faf4cd16d78
"locks/unix/thread_mutex.c:50") at memory/unix/apr_pools.c:1814
#15 0x7faf4cd07ce5 in apr_thread_mutex_create (mutex=0x225a798, flags=1, 
pool=0x225a710) at
locks/unix/thread_mutex.c:50
#16 0x7faf4cd0a164 in apr_pool_clear_debug (pool=0x225a710, file_line=0x488f09 
"mpm_fdqueue.c:236") at
memory/unix/apr_pools.c:1911
#17 0x0046c455 in ap_queue_info_push_pool (queue_info=0x22648b0, 
pool_to_recycle=0x225a710) at mpm_fdqueue.c:236
#18 0x7faf4bf18821 in process_lingering_close (cs=0x78d670) at event.c:1457
#19 0x7faf4bf196a8 in worker_thread (thd=0x6cae80, dummy=) at event.c:2083
#20 0x7faf4c68b5f0 in start_thread () from /lib64/libpthread.so.0
#21 0x7faf4c1f684d in clone () from /lib64/libc.so.6

So it seems a mutex gets created, which allocates memory, which in turn 
triggers debug logging, which walks pools and
finally tries to lock the not yet initialized lock.

Anyone aware of that? Any ideas how to fix?


This is strange. Before apr_thread_mutex_create is called by apr_pool_clear_debug 
pool->mutex is set to NULL. So IMHO in
frame #7 mutex should be NULL.
Which version of APR are you using?


1.7 with a few debug patches, that should really not make a difference here 
(but might offset line numbers a bit).
1.7.0, 1.7.x, 1.6.5 and 1.6.x do not differ in apr_pools.c.


I was looking at apr trunk. Maybe r1481186 fixes your issue.

Regards

Rüdiger

Re: crash during httpd cleanup when using APR debug library (APR_POOL_DEBUG)

[Rainer Jung]

Am 17.07.2019 um 10:03 schrieb Ruediger Pluem:



On 07/16/2019 11:28 PM, Rainer Jung wrote:

cross-posted to APR+HTTPD

Crahs happens in #2  0x7faf4c154945 in raise () from /lib64/libc.so.6
#3  0x7faf4c155f21 in abort () from /lib64/libc.so.6
#4  0x7faf4c14d810 in __assert_fail () from /lib64/libc.so.6
#5  0x7faf4c694219 in __pthread_tpp_change_priority () from 
/lib64/libpthread.so.0
#6  0x7faf4c68cd76 in __pthread_mutex_lock_full () from 
/lib64/libpthread.so.0
#7  0x7faf4cd07c29 in apr_thread_mutex_lock (mutex=0x2261fe0) at 
locks/unix/thread_mutex.c:108
#8  0x7faf4cd08603 in apr_pool_walk_tree (pool=0x225a710, fn=0x7faf4cd07fc0 
, data=0x7faf45777c90)
at memory/unix/apr_pools.c:1515
#9  0x7faf4cd08630 in apr_pool_walk_tree (pool=0x6a3ce0, fn=0x7faf4cd07fc0 
, data=0x7faf45777c90) at
memory/unix/apr_pools.c:1521
#10 0x7faf4cd08630 in apr_pool_walk_tree (pool=0x6a3770, fn=0x7faf4cd07fc0 
, data=0x7faf45777c90) at
memory/unix/apr_pools.c:1521
#11 0x7faf4cd08630 in apr_pool_walk_tree (pool=0x6a3110, fn=0x7faf4cd07fc0 
, data=0x7faf45777c90) at
memory/unix/apr_pools.c:1521
#12 0x7faf4cd086df in apr_pool_num_bytes (pool=0x6d81, recurse=) at memory/unix/apr_pools.c:2304
#13 0x7faf4cd0898f in apr_pool_log_event (pool=0x225a710, event=0x7faf4cd16e74 
"PCALLOC", file_line=0x7faf4cd16d78
"locks/unix/thread_mutex.c:50", deref=-1)
 at memory/unix/apr_pools.c:1543
#14 0x7faf4cd098b8 in apr_pcalloc_debug (pool=0x225a710, size=64, 
file_line=0x7faf4cd16d78
"locks/unix/thread_mutex.c:50") at memory/unix/apr_pools.c:1814
#15 0x7faf4cd07ce5 in apr_thread_mutex_create (mutex=0x225a798, flags=1, 
pool=0x225a710) at
locks/unix/thread_mutex.c:50
#16 0x7faf4cd0a164 in apr_pool_clear_debug (pool=0x225a710, file_line=0x488f09 
"mpm_fdqueue.c:236") at
memory/unix/apr_pools.c:1911
#17 0x0046c455 in ap_queue_info_push_pool (queue_info=0x22648b0, 
pool_to_recycle=0x225a710) at mpm_fdqueue.c:236
#18 0x7faf4bf18821 in process_lingering_close (cs=0x78d670) at event.c:1457
#19 0x7faf4bf196a8 in worker_thread (thd=0x6cae80, dummy=) at event.c:2083
#20 0x7faf4c68b5f0 in start_thread () from /lib64/libpthread.so.0
#21 0x7faf4c1f684d in clone () from /lib64/libc.so.6

So it seems a mutex gets created, which allocates memory, which in turn 
triggers debug logging, which walks pools and
finally tries to lock the not yet initialized lock.

Anyone aware of that? Any ideas how to fix?


This is strange. Before apr_thread_mutex_create is called by apr_pool_clear_debug 
pool->mutex is set to NULL. So IMHO in
frame #7 mutex should be NULL.
Which version of APR are you using?


1.7 with a few debug patches, that should really not make a difference 
here (but might offset line numbers a bit). 1.7.0, 1.7.x, 1.6.5 and 
1.6.x do not differ in apr_pools.c.


The code that jumps into the mutex creation is:

1877 APR_DECLARE(void) apr_pool_clear_debug(apr_pool_t *pool,
1878const char *file_line)
1879 {
...
1902
1903 pool_clear_debug(pool, file_line);
1904
1905 #if APR_HAS_THREADS
1906 /* If we had our own mutex, it will have been destroyed by
1907  * the registered cleanups.  Recreate the mutex.  Unlock
1908  * the mutex we obtained above.
1909  */
1910 if (mutex != pool->mutex) {
1911 (void)apr_thread_mutex_create(&pool->mutex,
1912   APR_THREAD_MUTEX_NESTED, pool);
1913
1914 if (mutex != NULL)
1915 (void)apr_thread_mutex_unlock(mutex);
1916 }
1917 #endif /* APR_HAS_THREADS */
1918 }

and in pool_clear_debug I see overwriting of pool allocated bytes with a 
'A' to poison data but I don't see, where pool->mutex is nulled.


Here's data inspection from gdb for pool, pool->parent (pconf), 
pool->mutex and pool->parent->mutex. IMHO one can see, that pool->mutex 
has indeed been treated by poison 'A'.


(gdb) print *pool
$1 = {parent = 0x6a3ce0, child = 0x0, sibling = 0x7729b0, ref = 
0x6a3ce8, cleanups = 0x0, free_cleanups = 0x0, allocator = 0x784a20, 
subprocesses = 0x0,
  abort_fn = 0x437920 , user_data = 0x0, tag = 
0x7faf4bf1d62a "transaction", joined = 0x0, nodes = 0x9335b0, file_line 
= 0x7faf4bf1d61d "event.c:1829",
  creation_flags = 0, stat_alloc = 1, stat_total_alloc = 45, stat_clear 
= 1, owner = 140390560073488, mutex = 0x2261fe0, pre_cleanups = 0x0}


(gdb) print *pool->parent
$2 = {parent = 0x6a3770, child = 0x225a710, sibling = 0x0, ref = 
0x6c2c70, cleanups = 0x0, free_cleanups = 0x0, allocator = 0x0, 
subprocesses = 0x0,
  abort_fn = 0x437920 , user_data = 0x7911d0, tag = 
0x47cb57 "pconf", joined = 0x0, nodes = 0x227eaf0, file_line = 0x47cb4c 
"main.c:296", creation_flags = 0,
  stat_alloc = 86248, stat_total_alloc = 172527, stat_clear = 1, owner 
= 140390893647616, mutex = 0

Re: apr_pool_join() and APR_POOL_DEBUG

[Rainer Jung]

Am 17.07.2019 um 01:30 schrieb Branko Čibej:

On 16.07.2019 23:18, Rainer Jung wrote:

I had some need for using APR_POOL_DEBUG today and ran into a
situation where pool lifetimes needed a hint using apr_pool_join().
That is all documented and fine, except that I was surprised to see,
that apr_pool_join() doesn't work unless the application itself is
also compiled with APR_POOL_DEBUG. I think that's not the intend, one
should be able to replace the apr library against a debug one during
lifetime.

So IMHO apr_pool_join() in apr has to be a noop function when
APR_POOL_DEBUG is not defined, so that the function call is contained
in the application.

Currently when the app is compiled without APR_POOL_DEBUG, then in
apr.h we only have

#if APR_POOL_DEBUG || defined(DOXYGEN)
...
#else /* APR_POOL_DEBUG or DOXYGEN */

#ifdef apr_pool_join
#undef apr_pool_join
#endif
#define apr_pool_join(a,b)
...
#endif /* APR_POOL_DEBUG or DOXYGEN */

so apr_pool_join() would be an empty macro.

Other pool debug functions seem to share the same fate:

- apr_pool_owner_set()
- apr_pool_find()
- apr_pool_num_bytes()
- apr_pool_lock()
- and as mentioned apr_pool_join()

Any opinion on whether we should introduce empty implementations
instead of empty macros for those all?


What should apr_pool_join and the other pool debugging functions do if
you're not debugging pools? They should be no-ops. So replacing them
with empty macros makes sense.

Or are you arguing that two libapr variants, one compiles with
APR_POOL_DEBUG and one without, should be ABI- compatible?


As far as I understand it, one should be able to replace the normal apr 
library by a debug apr library at runtime and get pool debugging. There 
should be no need to provide a apr debug compilation of the app itself.


If an app needs calls to apr_pool_join() to guide the debugging, they 
must be in the source code of the app and must not get compiled out.


No-op would only be OK, if the app they are function calls to no-op 
functions, but not if they get compiled out.


So yes, I think it must be ABI compatible and except for the above 5 
functions it is.


Regards,

Rainer

crash during httpd cleanup when using APR debug library (APR_POOL_DEBUG)

[Rainer Jung]

cross-posted to APR+HTTPD

Crahs happens in #2  0x7faf4c154945 in raise () from /lib64/libc.so.6
#3  0x7faf4c155f21 in abort () from /lib64/libc.so.6
#4  0x7faf4c14d810 in __assert_fail () from /lib64/libc.so.6
#5  0x7faf4c694219 in __pthread_tpp_change_priority () from 
/lib64/libpthread.so.0
#6  0x7faf4c68cd76 in __pthread_mutex_lock_full () from 
/lib64/libpthread.so.0
#7  0x7faf4cd07c29 in apr_thread_mutex_lock (mutex=0x2261fe0) at 
locks/unix/thread_mutex.c:108
#8  0x7faf4cd08603 in apr_pool_walk_tree (pool=0x225a710, 
fn=0x7faf4cd07fc0 , data=0x7faf45777c90) at 
memory/unix/apr_pools.c:1515
#9  0x7faf4cd08630 in apr_pool_walk_tree (pool=0x6a3ce0, 
fn=0x7faf4cd07fc0 , data=0x7faf45777c90) at 
memory/unix/apr_pools.c:1521
#10 0x7faf4cd08630 in apr_pool_walk_tree (pool=0x6a3770, 
fn=0x7faf4cd07fc0 , data=0x7faf45777c90) at 
memory/unix/apr_pools.c:1521
#11 0x7faf4cd08630 in apr_pool_walk_tree (pool=0x6a3110, 
fn=0x7faf4cd07fc0 , data=0x7faf45777c90) at 
memory/unix/apr_pools.c:1521
#12 0x7faf4cd086df in apr_pool_num_bytes (pool=0x6d81, 
recurse=) at memory/unix/apr_pools.c:2304
#13 0x7faf4cd0898f in apr_pool_log_event (pool=0x225a710, 
event=0x7faf4cd16e74 "PCALLOC", file_line=0x7faf4cd16d78 
"locks/unix/thread_mutex.c:50", deref=-1)

at memory/unix/apr_pools.c:1543
#14 0x7faf4cd098b8 in apr_pcalloc_debug (pool=0x225a710, size=64, 
file_line=0x7faf4cd16d78 "locks/unix/thread_mutex.c:50") at 
memory/unix/apr_pools.c:1814
#15 0x7faf4cd07ce5 in apr_thread_mutex_create (mutex=0x225a798, 
flags=1, pool=0x225a710) at locks/unix/thread_mutex.c:50
#16 0x7faf4cd0a164 in apr_pool_clear_debug (pool=0x225a710, 
file_line=0x488f09 "mpm_fdqueue.c:236") at memory/unix/apr_pools.c:1911
#17 0x0046c455 in ap_queue_info_push_pool (queue_info=0x22648b0, 
pool_to_recycle=0x225a710) at mpm_fdqueue.c:236
#18 0x7faf4bf18821 in process_lingering_close (cs=0x78d670) at 
event.c:1457
#19 0x7faf4bf196a8 in worker_thread (thd=0x6cae80, dummy=optimized out>) at event.c:2083

#20 0x7faf4c68b5f0 in start_thread () from /lib64/libpthread.so.0
#21 0x7faf4c1f684d in clone () from /lib64/libc.so.6

So it seems a mutex gets created, which allocates memory, which in turn 
triggers debug logging, which walks pools and finally tries to lock the 
not yet initialized lock.


Anyone aware of that? Any ideas how to fix?

Thanks and regards,

Rainer

apr_pool_join() and APR_POOL_DEBUG

[Rainer Jung]

I had some need for using APR_POOL_DEBUG today and ran into a situation 
where pool lifetimes needed a hint using apr_pool_join(). That is all 
documented and fine, except that I was surprised to see, that 
apr_pool_join() doesn't work unless the application itself is also 
compiled with APR_POOL_DEBUG. I think that's not the intend, one should 
be able to replace the apr library against a debug one during lifetime.


So IMHO apr_pool_join() in apr has to be a noop function when 
APR_POOL_DEBUG is not defined, so that the function call is contained in 
the application.


Currently when the app is compiled without APR_POOL_DEBUG, then in apr.h 
we only have


#if APR_POOL_DEBUG || defined(DOXYGEN)
...
#else /* APR_POOL_DEBUG or DOXYGEN */

#ifdef apr_pool_join
#undef apr_pool_join
#endif
#define apr_pool_join(a,b)
...
#endif /* APR_POOL_DEBUG or DOXYGEN */

so apr_pool_join() would be an empty macro.

Other pool debug functions seem to share the same fate:

- apr_pool_owner_set()
- apr_pool_find()
- apr_pool_num_bytes()
- apr_pool_lock()
- and as mentioned apr_pool_join()

Any opinion on whether we should introduce empty implementations instead 
of empty macros for those all?


Regards,

Rainer

Re: [result] [vote] Release apr-1.7.0 ?

[Rainer Jung]

Hi Dennis,

Am 04.04.2019 um 20:32 schrieb Dennis Clarke:

On 4/4/19 2:10 PM, William A Rowe Jr wrote:

First off, thanks all who have contributed to the 1.6.0 -> 1.7.0 evolution
in some large or small way. Secondly, thanks to all who reviewed.



May not matter much at this point but on Solaris 10 sparc I saw this in
tests :

us=$status; \
 progfailed="$progfailed '$prog mode
$mode'"; \
 fi; \
 done; \
 else \
 ./$prog -v; \
 status=$?; \
 if test $status != 0; then \
 teststatus=$status; \
 progfailed="$progfailed $prog"; \
 fi; \
 fi; \
done; \
if test $teststatus != 0; then \
 echo "Programs failed:$progfailed"; \
fi; \
exit $teststatus
ld.so.1: testlockperf: fatal: relocation error: file testlockperf:
symbol apr_thread_mutex_timedlock: referenced symbol not found
/bin/bash: line 2: 27221 Killed      ./$prog -v
.
.
.

However I think Rainer Jung had no such problems. I will go take a look
at my environment and see what went sideways.


Note that I am using GCC, you are using the Sun/Oracle compiler, but I 
expect that to not matter here. I also assume your "make" step worked, 
ie. it didn't end with an error.


Let's try to understand and track the root cause:

First:

% nm test/testlockperf | grep apr_thread_mutex_timedlock
[111]   |138752|   0|FUNC |GLOB |0|UNDEF 
|apr_thread_mutex_timedlock


Aha, so the symbol apr_thread_mutex_timedlock is indeen not defined in 
test/testlockperf. But since it can be executed on my environment, it 
must be provided by a library used by test/testlockperf. Ah, the symbol 
name starts with "apr" so it might be the apr library.


Let's look at the dependencies of the binary:

% elfdump -d test/testlockperf

Dynamic Section:  .dynamic
 index  tagvalue
   [0]  NEEDED0x2bd   libapr-1.so.0
   [1]  NEEDED0x2cb   libuuid.so.1
   [2]  NEEDED0x2d8   libsendfile.so.1
   [3]  NEEDED0x2e9   librt.so.1
   [4]  NEEDED0x2f4   libsocket.so.1
   [5]  NEEDED0x303   libnsl.so.1
   [6]  NEEDED0x30f   libpthread.so.1
   [7]  NEEDED0x2a7   libc.so.1
   [8]  INIT  0x1158c
   [9]  FINI  0x115a8
  [10]  RUNPATH   0x31f 
/path/to/my/build/dir/.libs:/path/to/my/install/dir/lib
  [11]  RPATH 0x31f 
/path/to/my/build/dir/.libs:/path/to/my/install/dir/lib

  [12]  HASH  0x100e8
  [13]  STRTAB0x105d4
  [14]  STRSZ 0x3bd
  [15]  SYMTAB0x10294
  [16]  SYMENT0x10
  [17]  CHECKSUM  0x2bac
  [18]  VERNEED   0x10994
  [19]  VERNEEDNUM0x1
  [20]  PLTRELSZ  0x120
  [21]  PLTREL0x7
  [22]  JMPREL0x10a08
  [23]  RELA  0x109b4
  [24]  RELASZ0x174
  [25]  RELAENT   0xc
  [26]  DEBUG 0
  [27]  FEATURE_1 0x1 [ PARINIT ]
  [28]  FLAGS 0   0
  [29]  FLAGS_1   0   0
  [30]  PLTGOT0x21d94
  [31]  NULL  0

Details may vary, but we see, that testlockperf want the libraries 
libapr-1.so.0, ..., libc.so.1 and will look for them in 
/path/to/my/build/dir/.libs and if not found in 
/path/to/my/install/dir/lib. Here names and paths get important.


The linker search paths for libraries can be influenced by setting eg. 
LD_LIBRARY_PATH. That will overwrite the search oath given in the 
binary. So if you eg. have set LD_LIBRARY_PATH and you have an older apr 
library (file libapr-1.so.0) there, e.g. a 1.6 version one, it might 
indeed not have all the symbols needed.


So lets first check, whether our new APR library exist in the search 
oath and whether it contains the needed symbold:


% ls -ld /path/to/my/build/dir/.libs/libapr-1.so.0
lrwxrwxrwx   1 jung kippdata  17 Apr  4 22:32 
/path/to/my/build/dir/.libs/libapr-1.so.0 -> libapr-1.so.0.7.0


Aha it is just a symlink to the versioned file. But remember, the linker 
looks for libapr-1.so.0, not libapr-1.so.0.7.0 (the former is the SONAME 
of the library). Lets check for the real file:


% ls -ld /path/to/my/build/dir/.libs/libapr-1.so*
lrwxrwxrwx   1 jung kippdata  17 Apr  4 22:32 
/path/to/my/build/dir/.libs/libapr-1.so -> libapr-1.so.0.7.0
lrwxrwxrwx   1 jung kippdata  17 Apr  4 22:32 
/path/to/my/build/dir/.libs/libapr-1.so.0 -> libapr-1.so.0.7.0
-rwx--   1 jung kippdata 1409256 Apr  2 23

Re: [vote] Release apr-1.7.0 ?

[Rainer Jung]

Am 01.04.2019 um 20:01 schrieb William A Rowe Jr:

Candidate tarballs are at the usual location;
https://apr.apache.org/dev/dist/

For the release of apr-1.7.0
   [X]   +1 looks great!
   [  ]  -1 something is broken

This vote will conclude April 4th 2pm EDT, for potential
announcement Friday.


+1 for releasse.

Detailed APR test results:

- files signed, checksums correct

- svn compared with gz, bz2 and zip only minor differences
  (libtool m4 files in gz and bz2, which seem to not get
   cleaned up by buildconf)

- I built and made check on the following platforms:
  - Solaris 10 Sparc, gcc 8.2.0
  - SuSE Linux Enterprise 11 and 12 64 Bit
  - RedHat Enterprise Linux 6 and 7 64 Bit

- config.guess timestamp='2019-01-01',
  pretty much up-to-date

- config.sub timestamp='2019-01-01'
  pretty much up-to-date

- all builds succeeded

- all "make check" ran fine, except for
  - binding to ::1 on Solaris with only IPv4 active,
not a regression:
  testsockets: Line 131: Could not bind socket (126):
  Cannot assign requested address
  Line 189: Condition is false, but expected true
  FAILED 1 of 7

- Some warnings during "make check":

  - Solaris 10 (not a regression)
  testsock: Line 433: Cannot test if connect completes synchronously
  SUCCESS

  - Solaris 10, OK doesn't know how to cork
  testsockopt: Line 84: TCP isn't corkable
  SUCCESS

  - Linux only 64 Bits: OK, no LFS needed on 64 Bit OS
  testlfs: Line 349: LFS support a no-op in 64-bit builds
  SUCCESS

I have not done any tests with the apr-util "make check" or the httpd 
test suite.


Thanks and regards,

Rainer

Re: tarball.sha256 file formatting (Was Re: [VOTE] apr-1.6.4 release?)

[Rainer Jung]

Hi Bill,

Am 01.04.2019 um 20:06 schrieb William A Rowe Jr:
On Mon, Sep 10, 2018 at 3:34 PM Rainer Jung <mailto:rainer.j...@kippdata.de>> wrote:


Am 07.09.2018 um 18:19 schrieb William A Rowe Jr:
 > Please cast your votes on the following release candidate
 > found at http://apr.apache.org/dev/dist/
 >
 > Release apr-1.6.4
 >    [  ] +/-1

One niggle: the sha files are not very tool friendly. I'm using
sha1sum/sha256sum/sha512sum which can check the checksum files but do
not understand the format used for 1.6.4. The previous version 1.6.3
used a checksum file format, the tools did understand. That format
would be

CHECKSUM *FILENAME

and the "*" (indicating binary) is only there if the file is not a text
file, eg. for our tarballs and zip files.


Hi Rainer,

I didn't find a follow-up on this, but I trust our tools/release.sh now 
gives you
an appropriate .sha256 format file your tools can parse, with the -r 
coreutils

flag passed to openssl?


The sha256 file currently available for 1.7.0 works well for me (using 
sha256sum -c).


Thanks and Regards,

Rainer

Re: svn commit: r1854603 - in /apr/apr-util/branches/1.7.x: build/apu-conf.m4 build/xml.m4 configure.in include/apu.h.in include/apu.hw include/apu.hwc xml/NWGNUmakefile xml/apr_xml.c xml/apr_xml_expa

[Rainer Jung]

Hi Nick,

Am 27.03.2019 um 23:42 schrieb Nick Kew:




On 27 Mar 2019, at 17:55, Rainer Jung  wrote:

This breaks 1.7 apu compilation using non-system expat for me. The expat 
detection sets INCLUDES but during compilation INCLUDES isn't used anywhere, so 
expat.h can't be found.


OK, I need to return to that.  Thanks for the reminder!

Does trunk compile for you in the same circumstances?
This is supposed to be a backport!


Trunk is APR which adds INCLUDES to EXTRA_INCLUDES and uses those during 
build. APU doesn't use EXTRA_INCLUDES anywhere but uses the 
APRUTIL_PRIV_INCLUDES and APRUTIL_INCLUDES variables, none of which the 
expat flags gets added to. dbd seems to use APRUTIL_PRIV_INCLUDES, all 
other dependencies seem to use APRUTIL_INCLUDES ...


Regards,

Rainer

Re: svn commit: r1854603 - in /apr/apr-util/branches/1.7.x: build/apu-conf.m4 build/xml.m4 configure.in include/apu.h.in include/apu.hw include/apu.hwc xml/NWGNUmakefile xml/apr_xml.c xml/apr_xml_expa

[Rainer Jung]

This breaks 1.7 apu compilation using non-system expat for me. The expat 
detection sets INCLUDES but during compilation INCLUDES isn't used 
anywhere, so expat.h can't be found.


Regards,

Rainer

Am 01.03.2019 um 17:07 schrieb n...@apache.org:

Author: niq
Date: Fri Mar  1 16:07:44 2019
New Revision: 1854603

URL: http://svn.apache.org/viewvc?rev=1854603&view=rev
Log:
apr_xml: backport libxml2 support (r1084621) and build (r1085350)
with their subsequent minor patches.
First pass: this requires further followup work to backport subsequent
patches dealing with non-*X platform builds (not to mention testing).

Added:
 apr/apr-util/branches/1.7.x/build/xml.m4
 apr/apr-util/branches/1.7.x/xml/apr_xml_expat.c
 apr/apr-util/branches/1.7.x/xml/apr_xml_internal.h
 apr/apr-util/branches/1.7.x/xml/apr_xml_libxml2.c
Modified:
 apr/apr-util/branches/1.7.x/build/apu-conf.m4
 apr/apr-util/branches/1.7.x/configure.in
 apr/apr-util/branches/1.7.x/include/apu.h.in
 apr/apr-util/branches/1.7.x/include/apu.hw
 apr/apr-util/branches/1.7.x/include/apu.hwc
 apr/apr-util/branches/1.7.x/xml/NWGNUmakefile
 apr/apr-util/branches/1.7.x/xml/apr_xml.c

Modified: apr/apr-util/branches/1.7.x/build/apu-conf.m4
URL: 
http://svn.apache.org/viewvc/apr/apr-util/branches/1.7.x/build/apu-conf.m4?rev=1854603&r1=1854602&r2=1854603&view=diff
==
--- apr/apr-util/branches/1.7.x/build/apu-conf.m4 (original)
+++ apr/apr-util/branches/1.7.x/build/apu-conf.m4 Fri Mar  1 16:07:44 2019
@@ -46,120 +46,6 @@ AC_DEFUN([APU_FIND_APR], [
AC_SUBST(APR_BUILD_DIR)
  ])
  
-dnl

-dnl APU_TRY_EXPAT_LINK(
-dnl  test-message, cache-var-name, hdrs, libs,
-dnl  [actions-on-success], [actions-on-failure])
-dnl
-dnl Tests linking against expat with libraries 'libs' and includes
-dnl 'hdrs', passing message + cache-var-name to AC_CACHE_CHECK.
-dnl On success, sets $expat_libs to libs, sets $apu_have_expat to 1,
-dnl and runs actions-on-success; on failure runs actions-on-failure.
-dnl
-AC_DEFUN([APU_TRY_EXPAT_LINK], [
-AC_CACHE_CHECK([$1], [$2], [
-  apu_expat_LIBS=$LIBS
-  LIBS="$LIBS $4"
-  AC_TRY_LINK([#include 
-#include <$3>], [XML_ParserCreate(NULL);],
-[$2=yes], [$2=no])
-  LIBS=$apu_expat_LIBS
-])
-
-if test $[$2] = yes; then
-   AC_DEFINE([HAVE_]translit([$3], [a-z./], [A-Z__]), 1,
- [Define if $3 is available])
-   apu_expat_libs="$4"
-   apu_has_expat=1
-   $5
-else
-   apu_has_expat=0
-   $6
-fi
-])
-
-dnl
-dnl APU_SYSTEM_EXPAT: tests for a system expat installation
-dnl If present, sets $apu_has_expat to 1 and adjusts LDFLAGS/CPPFLAGS
-dnl appropriately.  This is mostly for compatibility with existing
-dnl expat releases; all but the first APU_TRY_EXPAT_LINK call could
-dnl be dropped later.
-dnl
-AC_DEFUN([APU_SYSTEM_EXPAT], [
-
-  APU_TRY_EXPAT_LINK([Expat 1.95.x], apu_cv_expat_system,
-[expat.h], [-lexpat])
-
-  if test $apu_has_expat = 0; then
-APU_TRY_EXPAT_LINK([old Debian-packaged expat], apu_cv_expat_debian,
-   [xmltok/xmlparse.h], [-lxmlparse -lxmltok])
-  fi
-
-  if test $apu_has_expat = 0; then
-APU_TRY_EXPAT_LINK([old FreeBSD-packaged expat], apu_cv_expat_freebsd,
-   [xml/xmlparse.h], [-lexpat])
-  fi
-
-  if test $apu_has_expat = 0; then
-APU_TRY_EXPAT_LINK([Expat 1.0/1.1], apu_cv_expat_1011,
-   [xmlparse/xmlparse.h], [-lexpat])
-  fi
-
-  if test $apu_has_expat = 0; then
-APR_ADDTO(LDFLAGS, [-L/usr/local/lib])
-APR_ADDTO(CPPFLAGS, [-I/usr/local/include])
-
-APU_TRY_EXPAT_LINK([Expat 1.95.x in /usr/local],
-   apu_cv_expat_usrlocal, [expat.h], [-lexpat],
-   [APR_ADDTO(APRUTIL_INCLUDES, [-I/usr/local/include])
-APR_ADDTO(APRUTIL_LDFLAGS, [-L/usr/local/lib])],[
-   APR_REMOVEFROM(LDFLAGS, [-L/usr/local/lib])
-   APR_REMOVEFROM(CPPFLAGS, [-I/usr/local/include])
-  ])
-  fi
-])
-
-
-dnl
-dnl APU_FIND_EXPAT: figure out where EXPAT is located
-dnl
-AC_DEFUN([APU_FIND_EXPAT], [
-
-save_cppflags="$CPPFLAGS"
-save_ldflags="$LDFLAGS"
-
-apu_has_expat=0
-
-apu_try_external_expat=1
-
-AC_ARG_WITH([expat],
-[  --with-expat=DIRspecify Expat location], [
-  if test "$withval" = "yes"; then
-AC_MSG_ERROR([a directory must be specified for --with-expat])
-  elif test "$withval" = "no"; then
-AC_MSG_ERROR([Expat cannot be disabled (at this time)])
-  else
-# Add given path to standard search paths if appropriate:
-if test "$withval" != "/usr"; then
-  APR_ADDTO(LDFLAGS, [-L$withval/lib])
-  APR_ADDTO(CPPFLAGS, [-I$withval/include])
-  APR_ADDTO(APRUTIL_INCLUDES, [-I$withval/include])
-  APR_ADDTO(APRUTIL_LDFLAGS, [-L$withval/lib])
-fi
-  fi
-])
-
-if test $apu_try_external_expat = 1; then
-  APU_SYSTEM_EXPAT
-fi
-
-APR_ADDTO(APRUTIL_EXPORT_LIBS, [$apu_expat_libs])
-APR_ADDTO(APRUTIL_LIBS, [$apu_expat_libs])
-
-CPPFLAGS=$save_cppflags
-LDFLAGS=$save_ldflags
-])
-
  
  dnl

  dnl Find a particular LDAP l

Re: svn commit: r1838963 - /apr/apr/branches/1.6.x/poll/unix/port.c

[Rainer Jung]

Am 25.08.2018 um 02:20 schrieb Nick Kew:



On 25 Aug 2018, at 00:39, Yann Ylavic  wrote:

On Fri, Aug 24, 2018 at 11:13 PM  wrote:


-if ((*num = j)) { /* any event besides wakeup pipe? */
+if (nres > 0) { /* any event besides wakeup pipe? */
+*num = nres;
 rv = APR_SUCCESS;


Shouldn't we set *num = 0 still?


It already is (at the top).  But yes, that could be further rationalised.

I basically fixed what caused me to take twice as long as it should
have done to review it (though obviously the ugliness was older
than your fix I was reviewing).


Btw, this commit probably needs to go to trunk too.


Agreed.  And 1.7.  Will do - unless you get there first.


Did this (apply to trunk+1.7) happen?

Regards,

Rainer

Re: [VOTE] apr-1.6.4 release?

[Rainer Jung]

Am 07.09.2018 um 18:19 schrieb William A Rowe Jr:

Please cast your votes on the following release candidate
found at http://apr.apache.org/dev/dist/

Release apr-1.6.4
   [  ] +/-1

This vote will conclude at 1pm EDT Monday 9/10, for
announcement Tuesday in the event of a successful vote.


Would have been +1 for me, except for the now applied win32 fix.

One niggle: the sha files are not very tool friendly. I'm using 
sha1sum/sha256sum/sha512sum which can check the checksum files but do 
not understand the format used for 1.6.4. The previous version 1.6.3 
used a checksum file format, the tools did understand. That format would be


CHECKSUM *FILENAME

and the "*" (indicating binary) is only there if the file is not a text 
file, eg. for our tarballs and zip files.


Regards,

Rainer

Re: [VOTE] apr-1.6.4 release?

[Rainer Jung]

Correct, r1839769 plus backports are the culprit. Looks like "tm->" must 
be replaced by "(*ostime)->" or similar in line 302:


if ((*ostime)->wMonth < 1 || (*ostime)->wMonth > 12)

I currently only have access to svn viewvc, so can't apply right now.

Regards,

Rainer

Re: svn commit: r1834513 - in /apr/apr/branches/1.6.x: STATUS file_io/win32/buffer.c

[Rainer Jung]

Any progress on this? A 1.6 release seems to be very close but I don't 
know whether th 1.6 code is reasonable for release as it is right now.


Regards,

Rainer

Am 29.06.2018 um 09:49 schrieb jean-frederic clere:

On 27/06/18 19:08, William A Rowe Jr wrote:

Any reason this is absent from development branch 1.7.x? I think all such
1.x-only issues should be started there. (Similarly with apr-util.)


Oops my bad :-( Thanks!
and I need to check if the trunk code can be use there:
file->flags & APR_FOPEN_XTHREAD instead file->mutex

Cheers

Jean-Frederic



On Wed, Jun 27, 2018 at 10:21 AM, mailto:jfcl...@apache.org>> wrote:

 Author: jfclere
 Date: Wed Jun 27 15:21:19 2018
 New Revision: 1834513

 URL: http://svn.apache.org/viewvc?rev=1834513&view=rev
 
 Log:
 remove the wrong proposal in STATUS and commit a fix for my crash
 problems.

 Modified:
     apr/apr/branches/1.6.x/STATUS
     apr/apr/branches/1.6.x/file_io/win32/buffer.c

 Modified: apr/apr/branches/1.6.x/STATUS
 URL:
 
http://svn.apache.org/viewvc/apr/apr/branches/1.6.x/STATUS?rev=1834513&r1=1834512&r2=1834513&view=diff
 

 
==
 --- apr/apr/branches/1.6.x/STATUS [utf-8] (original)
 +++ apr/apr/branches/1.6.x/STATUS [utf-8] Wed Jun 27 15:21:19 2018
 @@ -98,10 +98,6 @@ CURRENT VOTES:
        1.5.x patch:
 http://people.apache.org/~jim/patches/apr-1.5-permset.patch
 
        +1:

 -    * make sure we don't unlock mutex when we haven't locked it.
 -      1.6.x patch:
 http://people.apache.org/~jfclere/patch.180618.txt
 
 -      +1 jfclere
 -
  CURRENT test/testall -v EXCEPTIONS:

      Please add any platform anomilies to the following exception list.

 Modified: apr/apr/branches/1.6.x/file_io/win32/buffer.c
 URL:
 
http://svn.apache.org/viewvc/apr/apr/branches/1.6.x/file_io/win32/buffer.c?rev=1834513&r1=1834512&r2=1834513&view=diff
 

 
==
 --- apr/apr/branches/1.6.x/file_io/win32/buffer.c (original)
 +++ apr/apr/branches/1.6.x/file_io/win32/buffer.c Wed Jun 27
 15:21:19 2018
 @@ -23,13 +23,17 @@ APR_DECLARE(apr_status_t) apr_file_buffe
  {
      apr_status_t rv;

 -    apr_thread_mutex_lock(file->mutex);
 +    if (file->mutex) {
 +        apr_thread_mutex_lock(file->mutex);
 +    }

      if(file->buffered) {
          /* Flush the existing buffer */
          rv = apr_file_flush(file);
          if (rv != APR_SUCCESS) {
 -            apr_thread_mutex_unlock(file->mutex);
 +            if (file->mutex) {
 +                apr_thread_mutex_unlock(file->mutex);
 +            }
              return rv;
          }
      }
 @@ -48,7 +52,9 @@ APR_DECLARE(apr_status_t) apr_file_buffe
              file->buffered = 0;
      }

 -    apr_thread_mutex_unlock(file->mutex);
 +    if (file->mutex) {
 +        apr_thread_mutex_unlock(file->mutex);
 +    }

      return APR_SUCCESS;
  }

Re: 1.6 release?

[Rainer Jung]

I think the following, especially the part about 1834513 is still unhandled.

Regards,

Rainer

Am 25.08.2018 um 15:22 schrieb Rainer Jung:

Am 24.08.2018 um 23:41 schrieb Nick Kew:

On Fri, 24 Aug 2018 09:16:54 -0400
Eric Covener  wrote:


Starting a new thread as potential RM's may be filtering bugzilla
emails.

There are a lot of reports of PR62644 from solaris users of httpd, can
anyone RM?


You've spurred me into reviewing changes in svn since last release.
There seems to be not much to see other than the fix for the
solaris poll problem.

There's quite a bit more in trunk (some of it annoyingly
not labelled in commit messages).  I've gone briefly through
that and see no obvious backports.  Anyone else?


Some inconsistencies between trunk / 1.7 and 1.6 seen by looking at 
file_io/win32/buffer.c but the original commits also touch other files:


% diff -u apr/apr/trunk/file_io/win32/buffer.c 
apr/apr/branches/1.6.x/file_io/win32/buffer.c
--- apr/apr/trunk/file_io/win32/buffer.c    2018-08-25 
15:05:20.124591000 +
+++ apr/apr/branches/1.6.x/file_io/win32/buffer.c   2018-07-28 
15:35:43.382858000 +

@@ -23,7 +23,7 @@
  {
  apr_status_t rv;

-    if (file->flags & APR_FOPEN_XTHREAD) {
+    if (file->mutex) {
  apr_thread_mutex_lock(file->mutex);
  }

@@ -31,7 +31,7 @@
  /* Flush the existing buffer */
  rv = apr_file_flush(file);
  if (rv != APR_SUCCESS) {
-    if (file->flags & APR_FOPEN_XTHREAD) {
+    if (file->mutex) {
  apr_thread_mutex_unlock(file->mutex);
  }
  return rv;
@@ -52,7 +52,7 @@
  file->buffered = 0;
  }

-    if (file->flags & APR_FOPEN_XTHREAD) {
+    if (file->mutex) {
  apr_thread_mutex_unlock(file->mutex);
  }

Introduced by r1808457 and 1829962. The first one also touches 
file_io/win32/open.c and file_io/win32/readwrite.c in similar ways.


Then:

% diff -u apr/apr/branches/1.6.x/file_io/win32/buffer.c 
apr/apr/branches/1.7.x/file_io/win32/buffer.c
--- apr/apr/branches/1.6.x/file_io/win32/buffer.c   2018-07-28 
15:35:43.382858000 +
+++ apr/apr/branches/1.7.x/file_io/win32/buffer.c   2017-09-25 
20:18:21.0 +

@@ -23,17 +23,13 @@
  {
  apr_status_t rv;

-    if (file->mutex) {
-    apr_thread_mutex_lock(file->mutex);
-    }
+    apr_thread_mutex_lock(file->mutex);

  if(file->buffered) {
  /* Flush the existing buffer */
  rv = apr_file_flush(file);
  if (rv != APR_SUCCESS) {
-    if (file->mutex) {
-    apr_thread_mutex_unlock(file->mutex);
-    }
+    apr_thread_mutex_unlock(file->mutex);
  return rv;
  }
  }
@@ -52,9 +48,7 @@
  file->buffered = 0;
  }

-    if (file->mutex) {
-    apr_thread_mutex_unlock(file->mutex);
-    }
+    apr_thread_mutex_unlock(file->mutex);

  return APR_SUCCESS;
  }


I think that's due to some unfinished work by Jean-Frederic. See related 
but probably unfinished discussions on r1834513.


Regards,

Rainer

Re: svn commit: r1839755 - in /apr/apr/trunk: include/apr_json.h json/apr_json.c json/apr_json_decode.c

[Rainer Jung]

To clarify: the CI build failures only send a summary email to the dev 
list. If you want to see the output, you need to visit the CI build page 
(link included in the failure mail) and there clock on the appropriate 
"stdio" link. That's probably from where Bill took the log snippet.


Regards,

Rainer

Re: svn commit: r1834495 - in /apr/apr/branches/1.7.x: ./ build/ build/buildcheck.sh build/gen-build.py buildconf

[Rainer Jung]

Shouldn't this be backported to 1.6.x?

Am 27.06.2018 um 13:49 schrieb jor...@apache.org:

Author: jorton
Date: Wed Jun 27 11:49:33 2018
New Revision: 1834495

URL: http://svn.apache.org/viewvc?rev=1834495&view=rev
Log:
Merge r1834494 from trunk:

* build/buildcheck.sh, buildconf: Detect and run under Python 3 or 2,
   and respect $PYTHON.

* build/gen-build.py: Fix various Python 3 compatibility issues.


Modified:
 apr/apr/branches/1.7.x/   (props changed)
 apr/apr/branches/1.7.x/build/   (props changed)
 apr/apr/branches/1.7.x/build/buildcheck.sh
 apr/apr/branches/1.7.x/build/gen-build.py
 apr/apr/branches/1.7.x/buildconf

Propchange: apr/apr/branches/1.7.x/
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Wed Jun 27 11:49:33 2018
@@ -1,4 +1,4 @@
  /apr/apr/branches/1.4.x:1003369,1101301
-/apr/apr/trunk:733052,739635,741862,741866-741867,741869,741871,745763-745764,746310,747990,748080,748361,748371,748565,74,748902,748988,749810,760443,767895,775683,782838,783398,783958,784633,784773,788588,789050,793192-793193,794118,794485,795267,799497,800627,809745,809854,810472,811455,813063,821306,829490,831641,832904,835607,888669,892028,892159,892435,892909,896382,896653,899905,901088,902077,902090,908427,910419,910597,917819,917837-917838,923311,923320,925965,929796,930508,931973,932585,951771,960665,960671,979891,983618,989450,990435,1003338,100,107,1055657,1072165,1078845,1081462,1081495,1083038,1083242,1084662,1086695,1088023,1089031,1089129,1089438,1099348,1103310,1183683,1183685-1183686,1183688,1183693,1183698,1213382,1235047,1236970,1237078,1237507,1240472,1340286,1340288,1340470,1341193,1341196,1343233,1343243,1367050,1368819,1370494,1372018,1372022,1372093,1372849,1376957,1384764,1389077,1400200,1402868,1405985,1406690,1420106,1420109,1425356,1428809,143
  
8940,1438957-1438959,1442903,1449568,1456418,1459994,1460179-1460180,1460241,1460399,1460405,1462738,1462813,1470186,1470348,1475509,1478905,1480067,1481262,1481265,1484271,1487796,1489517,1496407,1502804,1510354,1516261,1523384,1523479,1523484,1523505,1523521,1523604,1523613,1523615,1523844-1523845,1523853,1524014,1524031,1528797,1528809,1529488,1529495,1529515,1529521,1529668,1530786,1530800,1530988,1531554,1531768,1531884,1532022,1533104,1533111,1533979,1535027,1535157,1536744,1538171,1539374,1539389,1539455,1539603,1541054,1541061,1541486,1541655,1541666,1541744,1542601,1542779,1543033,1543056,1548575,1550907,1551650,1551659,1558905,1559382,1559873,1559975,1561040,1561260,1561265,1561321,1561347,1561356,1561361,1561394,1561555,1571894,1575509,1578420,1587045,1587063,1587543,1587545,1588878,1588937,1589982,1593611,1593614-1593615,1593680,1594684,1594708,1595549,1597797,1597803,1604590,1604596,1604598,1605104,1610854,1611023,1611107,160,167,1611120,1611125,1611184,1611193,
  
1611466,1611515,1611517,1625173,1626564,1634615,1642159,1648830,1664406,1664447,1664451,1664471,1664769-1664770,1664775,1664904,1664911,1664958,1666341,1666411,1666458,111,1667420-1667421,1667423,1667900-1667901,1667903,1667914-1667916,1667962,1669077,1671292,1671329,1671356,1671386,1671389,1671513-1671514,1671957,1672354,1672366,1672495,1672575,1675644,1675656,1675668,1676013,1683521,1685929,1696140,1696767,1722547,1722557,1726928,1727020,1727160,1727175,1727199,1728957,1732582,1733451,1733594,1733694,1733706,1733708,1733775,1734816,1736552,1738791,1738925,1750374,1755709,1755740,1755746,1755758,1755954,1761279,1762326,1774712,1774973,1775069,1776994,1776998,1788334,1788337,1788929,1789947,1789998,1790045,1790200,1790296,1790302-1790304,1790330-1790331,1790436,1790439,1790444,1790446,1790488,1790521,1790523,1790569,1790632,1791598,1791718,1791728,1792621-1792622,1792625,1792961,1792963,1797415,1798105,1805380,1808039,1808836,1808910,1809649,1810452,1813286,1813330,1814239-18142
  
40,1814326,1814329,1814331,1816527,1816628,1817485,1819857-1819858,1819860-1819861,1819934-1819935,1820080,1820755,1822357,1827534,1832691
+/apr/apr/trunk:733052,739635,741862,741866-741867,741869,741871,745763-745764,746310,747990,748080,748361,748371,748565,74,748902,748988,749810,760443,767895,775683,782838,783398,783958,784633,784773,788588,789050,793192-793193,794118,794485,795267,799497,800627,809745,809854,810472,811455,813063,821306,829490,831641,832904,835607,888669,892028,892159,892435,892909,896382,896653,899905,901088,902077,902090,908427,910419,910597,917819,917837-917838,923311,923320,925965,929796,930508,931973,932585,951771,960665,960671,979891,983618,989450,990435,1003338,100,107,1055657,1072165,1078845,1081462,1081495,1083038,1083242,1084662,1086695,1088023,1089031,1089129,1089438,1099348,1103310,1183683,1183685-1183686,1183688,1183693,1183698,1213382,1235047,1236970,1237078,1237507,1240472,1340286,1340288,1340470,1341193,1341196,1343233,1343243,1367050,1368819,1370494,1372018,1372022,1372093,1372849,1376957,1384764,1389077,1400200,1402868,1405985,1

Backporting reslist changes?

[Rainer Jung]

I dont't remember the outcome of the discussions, but are any of the 
following three reslist changes reasonable for 1.6 backport?



r1834030 | ylavic | 2018-06-21 19:06:34 +0200 (Thu, 21 Jun 2018) | 6 lines

apr_reslist: always expire oldest entries first.

When a resource is to be acquired, we should check for expiring entries
starting from the oldest to the youngest one, otherwise it's only when
the latter expires that all of the resources are killed in a batch.


r1834061 | ylavic | 2018-06-21 23:49:40 +0200 (Thu, 21 Jun 2018) | 5 lines

apr_reslist: don't release/re-acquire the mutex in apr_reslist_release().

We can hold the mutex for the whole release time by adding and using an
unlocked reslist_maintain() helper, and ensure more fairness for the 
releaser.



r1834064 | ylavic | 2018-06-22 00:17:48 +0200 (Fri, 22 Jun 2018) | 3 lines

apr_reslist: put common code in push_resource().

Regards,

Rainer

Re: svn commit: r1836519 - /apr/apr/trunk/strings/apr_strings.c

[Rainer Jung]

Should this be changed or reverted? The discussion seems to have stalled.

And what about backport for 1.7.x and 1.6.x?

Regards,

Rainer

Am 24.07.2018 um 17:42 schrieb Yann Ylavic:

On Tue, Jul 24, 2018 at 12:53 AM, William A Rowe Jr  wrote:

I'm concerned that you've made a specific assumption of 2's compliment int.
Nothing in the spec or real world assures us of this. Intel x86 is 2's
compliment, but this is a bad assumption.


I doubt we support 1s complement archs but if so, something like this
would work:

static const char xtoa_digits[] = "9876543210123456789";
static const int xtoa_middle = (sizeof(xtoa_digits) - 1) / 2;

APR_DECLARE(char *) apr_itoa(apr_pool_t *p, int n)
{
 const int BUFFER_SIZE = sizeof(int) * 3 + 2;
 char *buf = apr_palloc(p, BUFFER_SIZE);
 char *start = buf + BUFFER_SIZE - 1;
 int negative = (n < 0);
 *start = 0;
 do {
 int tmp = n;
 n /= 10;
 *--start = xtoa_digits[xtoa_middle + tmp - n * 10];
 } while (n);
 if (negative) {
 *--start = '-';
 }
 return start;
}

Same for apr_ltoa() or apr_off_t_toa() (with the corresponding type
for "tmp"), no sign assumption.

Re: 1.6 release?

[Rainer Jung]

Am 24.08.2018 um 23:41 schrieb Nick Kew:

On Fri, 24 Aug 2018 09:16:54 -0400
Eric Covener  wrote:


Starting a new thread as potential RM's may be filtering bugzilla
emails.

There are a lot of reports of PR62644 from solaris users of httpd, can
anyone RM?


You've spurred me into reviewing changes in svn since last release.
There seems to be not much to see other than the fix for the
solaris poll problem.

There's quite a bit more in trunk (some of it annoyingly
not labelled in commit messages).  I've gone briefly through
that and see no obvious backports.  Anyone else?


Some inconsistencies between trunk / 1.7 and 1.6 seen by looking at 
file_io/win32/buffer.c but the original commits also touch other files:


% diff -u apr/apr/trunk/file_io/win32/buffer.c 
apr/apr/branches/1.6.x/file_io/win32/buffer.c
--- apr/apr/trunk/file_io/win32/buffer.c2018-08-25 
15:05:20.124591000 +
+++ apr/apr/branches/1.6.x/file_io/win32/buffer.c   2018-07-28 
15:35:43.382858000 +

@@ -23,7 +23,7 @@
 {
 apr_status_t rv;

-if (file->flags & APR_FOPEN_XTHREAD) {
+if (file->mutex) {
 apr_thread_mutex_lock(file->mutex);
 }

@@ -31,7 +31,7 @@
 /* Flush the existing buffer */
 rv = apr_file_flush(file);
 if (rv != APR_SUCCESS) {
-if (file->flags & APR_FOPEN_XTHREAD) {
+if (file->mutex) {
 apr_thread_mutex_unlock(file->mutex);
 }
 return rv;
@@ -52,7 +52,7 @@
 file->buffered = 0;
 }

-if (file->flags & APR_FOPEN_XTHREAD) {
+if (file->mutex) {
 apr_thread_mutex_unlock(file->mutex);
 }

Introduced by r1808457 and 1829962. The first one also touches 
file_io/win32/open.c and file_io/win32/readwrite.c in similar ways.


Then:

% diff -u apr/apr/branches/1.6.x/file_io/win32/buffer.c 
apr/apr/branches/1.7.x/file_io/win32/buffer.c
--- apr/apr/branches/1.6.x/file_io/win32/buffer.c   2018-07-28 
15:35:43.382858000 +
+++ apr/apr/branches/1.7.x/file_io/win32/buffer.c   2017-09-25 
20:18:21.0 +

@@ -23,17 +23,13 @@
 {
 apr_status_t rv;

-if (file->mutex) {
-apr_thread_mutex_lock(file->mutex);
-}
+apr_thread_mutex_lock(file->mutex);

 if(file->buffered) {
 /* Flush the existing buffer */
 rv = apr_file_flush(file);
 if (rv != APR_SUCCESS) {
-if (file->mutex) {
-apr_thread_mutex_unlock(file->mutex);
-}
+apr_thread_mutex_unlock(file->mutex);
 return rv;
 }
 }
@@ -52,9 +48,7 @@
 file->buffered = 0;
 }

-if (file->mutex) {
-apr_thread_mutex_unlock(file->mutex);
-}
+apr_thread_mutex_unlock(file->mutex);

 return APR_SUCCESS;
 }


I think that's due to some unfinished work by Jean-Frederic. See related 
but probably unfinished discussions on r1834513.


Regards,

Rainer

Re: 1.6 release?

[Rainer Jung]

Am 24.08.2018 um 23:41 schrieb Nick Kew:

On Fri, 24 Aug 2018 09:16:54 -0400
Eric Covener  wrote:


Starting a new thread as potential RM's may be filtering bugzilla
emails.

There are a lot of reports of PR62644 from solaris users of httpd, can
anyone RM?


You've spurred me into reviewing changes in svn since last release.
There seems to be not much to see other than the fix for the
solaris poll problem.

There's quite a bit more in trunk (some of it annoyingly
not labelled in commit messages).  I've gone briefly through
that and see no obvious backports.  Anyone else?


There's a list of changes done by Ivan Zhakov starting with 1785072 in 
early 2017 to improve Win32 file I/O performance. Maybe Ivan can tell, 
whether they can and should be backported to 1.7 (and 1.6?).


Regards,

Rainer

Re: svn commit: r1817889 - in /apr/apr/branches/1.7.x: ./ test/testipsub.c test/testlock.c

[Rainer Jung]

0179-1460180,1460241,1460399,1460405,1462738,1462813,1470186,1470348,1475509,1478905,1480067,1481262,1481265,1484271,1487796,1489517,1496407,1502804,1510354,1516261,1523384,1523479,1523484,1523505,1523521,1523604,1523613,1523615,1523844-1523845,1523853,1524014,1524031,1528797,1528809,1529488,1529495,1529515,1529521,1529668,1530786,1530800,1530988,1531554,1531768,1531884,1532022,1533104,1533111,1533979,1535027,1535157,1536744,1538171,1539374,1539389,1539455,1539603,1541054,1541061,1541486,1541655,1541666,1541744,1542601,1542779,1543033,1543056,1548575,1550907,1551650,1551659,1558905,1559382,1559873,1559975,1561040,1561260,1561265,1561321,1561347,1561356,1561361,1561394,1561555,1571894,1575509,1578420,1587045,1587063,1587543,1587545,1588878,1588937,1589982,1593611,1593614-1593615,1593680,1594684,1594708,1595549,1597797,1597803,1604590,1604596,1604598,1605104,1610854,1611023,1611107,160,167,1611120,1611125,1611184,1611193,
  
1611466,1611515,1611517,1625173,1626564,1634615,1642159,1648830,1664406,1664447,1664451,1664471,1664769-1664770,1664775,1664904,1664911,1664958,1666341,1666411,1666458,111,1667420-1667421,1667423,1667900-1667901,1667903,1667914-1667916,1667962,1669077,1671292,1671329,1671356,1671386,1671389,1671513-1671514,1671957,1672354,1672366,1672495,1672575,1675644,1675656,1675668,1676013,1683521,1685929,1696140,1696767,1722547,1722557,1726928,1727020,1727160,1727175,1727199,1728957,1732582,1733451,1733594,1733694,1733706,1733708,1733775,1734816,1736552,1738791,1738925,1750374,1755709,1755740,1755746,1755758,1755954,1761279,1762326,1774712,1774973,1775069,1776994,1776998,1788334,1788337,1788929,1789947,1789998,1790045,1790200,1790296,1790302-1790304,1790330-1790331,1790436,1790439,1790444,1790446,1790488,1790521,1790523,1790569,1790632,1791598,1791718,1791728,1792621-1792622,1792625,1792961,1792963,1797415,1798105,1805380,1808039,1808836,1808910,1809649,1810452,1813286,1813330,1816527,18166
  28,1817485
  /apr/apr/trunk/test/testnames.c:1460405
  /httpd/httpd/trunk:1604590

Modified: apr/apr/branches/1.7.x/test/testipsub.c
URL: 
http://svn.apache.org/viewvc/apr/apr/branches/1.7.x/test/testipsub.c?rev=1817889&r1=1817888&r2=1817889&view=diff
==
--- apr/apr/branches/1.7.x/test/testipsub.c (original)
+++ apr/apr/branches/1.7.x/test/testipsub.c Tue Dec 12 08:07:33 2017
@@ -173,18 +173,25 @@ static void test_parse_addr_port(abts_ca
  const char *addr, *scope_id;
  apr_port_t port;
  } *test, testcases[] = {
+/* Success cases */
  { "localhost:80", APR_SUCCESS, "localhost", NULL, 80 }
+,{ "localhost", APR_SUCCESS, "localhost", NULL, 0 }
  ,{ "www.example.com:8080", APR_SUCCESS, "www.example.com", NULL, 8080 
}
  ,{ "w:1", APR_SUCCESS, "w", NULL, 1 }
  ,{ "127.0.0.1:80", APR_SUCCESS, "127.0.0.1", NULL, 80 }
  ,{ "[::]:80", APR_SUCCESS, "::", NULL, 80 }
+,{ "[::%eth0]:80", APR_SUCCESS, "::", "eth0", 80 }
+,{ "[::%eth0]", APR_SUCCESS, "::", "eth0", 0 }
+,{ "8080", APR_SUCCESS, NULL, NULL, 8080 } /* API doc has this case */
+
+/* Failure cases */
  ,{ "localhost:99", APR_EINVAL, NULL, NULL, 0 }
  ,{ "localhost:0", APR_EINVAL, NULL, NULL, 0 }
+,{ "[abc]", APR_EINVAL, NULL, NULL, 0 }
  ,{ "[::]z:80", APR_EINVAL, NULL, NULL, 0 }
  ,{ "[:::80", APR_EINVAL, NULL, NULL, 0 }
  ,{ "[]:80", APR_EINVAL, NULL, NULL, 0 }
  ,{ "[::%]:80", APR_EINVAL, NULL, NULL, 0 }
-,{ "[::%eth0]:80", APR_SUCCESS, "::", "eth0", 80 }
  /*,{ "127.0.0.1:80x", APR_EINVAL, NULL, NULL, 0 }  <- should fail, 
doesn't  */
  /*,{ "127.0.0.1x:80", APR_EINVAL, NULL, NULL, 0 }  <- maybe should 
fail?, doesn't  */
  /*,{ "localhost:-1", APR_EINVAL, NULL, NULL, 0 }   <- should fail, 
doesn't */

Modified: apr/apr/branches/1.7.x/test/testlock.c
URL: 
http://svn.apache.org/viewvc/apr/apr/branches/1.7.x/test/testlock.c?rev=1817889&r1=1817888&r2=1817889&view=diff
==
--- apr/apr/branches/1.7.x/test/testlock.c (original)
+++ apr/apr/branches/1.7.x/test/testlock.c Tue Dec 12 08:07:33 2017
@@ -336,7 +336,7 @@ static void test_timeoutcond(abts_case *
  continue;
  }
  ABTS_INT_EQUAL(tc, 1, APR_STATUS_IS_TIMEUP(s));
-ABTS_ASSERT(tc, "Timer returned too late", end - begin - timeout < 
10);
+ABTS_ASSERT(tc, "Timer returned too late", end - begin - timeout < 
50);
  break;
  }
  ABTS_ASSERT(tc, "Too many retries", i < MAX_RETRY);




--
kippdata
informationstechnologie GmbH   Tel: 0228 98549 -0
Bornheimer Str. 33aFax: 0228 98549 -50
53111 Bonn www.kippdata.de

HRB 8018 Amtsgericht Bonn / USt.-IdNr. DE 196 457 417
Geschäftsführer: Dr. Thomas Höfer, Rainer Jung, Sven Maurmann

Re: Question on Unix Source for APR-util 1.6.1 and APR iconv 1.2.2

[Rainer Jung]

Am 29.06.2018 um 21:57 schrieb Duttera, Scott A CIV DISA SEL5 (US):

We are attempting to compile Apache 2.4 on a Unix system here, and got hit with 
prereqs by the compiler for the APR libraries.  When we downloaded the Unix 
Source for the APR-util 1.6.1 and APR iconv 1.2.2 software from the 
https://apr.apache.org/download.cgi web site, the README files say the software 
is for the Windows version.  Is this true, or is the documentation just wrong?  
If true, where can we get the Unix source for those to libraries?


To compile the Apache web server on a Unix system, you only need APR and 
APR-Util, but not APR-Iconv. Unix Sources for them are linked on the 
download page you cited.


Regards,

Rainer

Re: svn commit: r1833786 - /apr/apr/branches/1.6.x/STATUS

[Rainer Jung]

Am 21.06.2018 um 10:24 schrieb jean-frederic clere:

On 21/06/18 10:04, Yann Ylavic wrote:

On Thu, Jun 21, 2018 at 8:00 AM, jean-frederic clere  wrote:


OK if everyone agrees I will just commit my small fix and don't backport
the 2 other revisions.


Are we looking at the same 1.6.x code?
For me apr_file_buffer_set() locks the mutex unconditionally so it
must unlock it likewise, I don't understand what your patch solves.


that is what my patch is doing ;-)


Please be more explicit. I agree with Yann. From current unpatched APR 
1.6.x file_io/win32/buffer.c:


APR_DECLARE(apr_status_t) apr_file_buffer_set(apr_file_t *file,
  char * buffer,
  apr_size_t bufsize)
{
...
apr_thread_mutex_lock(file->mutex);

if(file->buffered) {
...
if (rv != APR_SUCCESS) {
apr_thread_mutex_unlock(file->mutex);
return rv;
}
}

... (no return)

apr_thread_mutex_unlock(file->mutex);

return APR_SUCCESS;
}

So where's the problem? Unconditional "apr_thread_mutex_lock" and 
unconditional "apr_thread_mutex_unlock" on every path before return.


Looking at your patch 
"http://people.apache.org/~jfclere/patches/patch.180618.txt"; (STATUS 
still contains a broken link, so we need to guess):


Index: file_io/win32/buffer.c
===
--- file_io/win32/buffer.c  (revision 1833783)
+++ file_io/win32/buffer.c  (working copy)
@@ -47,8 +47,10 @@
  */
 file->buffered = 0;
 }
-
-apr_thread_mutex_unlock(file->mutex);
+
+if (file->buffered) {
+apr_thread_mutex_unlock(file->mutex);
+}

 return APR_SUCCESS;
 }


any call to "apr_file_buffer_set" for a file with "buffered" not set 
will end with not giving back the lock.


Regards,

Rainer

Re: svn commit: r1819858 - /apr/apr/trunk/poll/unix/port.c

[Rainer Jung]

Hi Yann,

Am 03.01.2018 um 11:07 schrieb Yann Ylavic:

On Wed, Jan 3, 2018 at 5:00 AM, Rainer Jung  wrote:


I did a check on Solaris 10 Sparc. Looks good as well:


Thanks Rainer, the PR is about Sparc so this is quite valuable return.
I first thought it was some concurrency issues (in mpm_event) specific
to Solaris Sparc until I found this..



- r1819856 (without the additional check and without the fix): no hang in
testpoll

- r1819857 (with the additional check but still without the fix): hangs in
testpoll in pollset_wakeup, more precisely in the new for loop in iteration
2 of 2 and there in the call to apr_pollset_wakeup().


Don't you mean that the hang was in the second call to apr_pollset_poll()?


Yes, sorry.


I don't really expect apr_pollset_wakeup() to block unless the pipe is
full, which'd be a capacity of one byte only here...


The hang was in the following stack:

(gdb) bt full
#0  0xff0cbcb4 in _portfs () from /lib/libc.so.1
No symbol table info available.
#1  0xff046230 in port_getn () from /lib/libc.so.1
No symbol table info available.
#2  0xff374b18 in call_port_getn (port=61, list=0x1aa418, max=2, 
nget=0xffbff558, timeout=-1) at .../poll/unix/port.c:110

tv = {tv_sec = 0, tv_nsec = 0}
tvptr = 
ret = 
rv = 0
#3  0xff374d34 in impl_pollset_poll (pollset=0x1aa3a8, 
timeout=, num=0xffbff634, descriptors=0xffbff638)

at .../poll/unix/port.c:400
fd = 
ret = 
i = 
j = 
nget = 1
ep = 
rv = 
fp = 
#4  0xff3743ac in apr_pollset_poll (pollset=0x1aa3a8, timeout=-1, 
num=num@entry=0xffbff634, descriptors=descriptors@entry=0xffbff638)

at .../poll/unix/pollset.c:246
No locals.

=> So apr_pollset_poll()

#5  0x00028ffc in pollset_wakeup (tc=0xffbff6b0, data=0x0) at 
.../test/testpoll.c:805

rv = 
socket_pollfd = {p = 0xd1fd0, desc_type = APR_POLL_SOCKET, 
reqevents = 1, rtnevents = 1, desc = {f = 0x1c20c0, s = 0x1c20c0}, 
client_data = 0x1c20c0}

pollset = 0x1aa3a8
num = 0
descriptors = 0x0
i = 1

=> So i==1, second loop iteration

#6  0x000184d0 in abts_run_test (ts=ts@entry=0xd6dd8, f=0x28f64 
, value=value@entry=0x0) at .../test/abts.c:171

tc = {failed = 0, suite = 0xd6308}
ss = 0xd6308
#7  0x000292e4 in testpoll (suite=0xd6dd8) at .../test/testpoll.c:956
No locals.
#8  0x00046944 in main (argc=2, argv=) at .../test/abts.c:429
i = 
list_provided = 
suite = 0xd6dd8



- r1819858 (with the additional check and with the fix): no hang in testpoll

- r1819902 (head of trunk): no hang in testpoll

Thanks for finding and fixing this!

Happy new year,


Best wishes to all!


Regards,
Yann.

Re: svn commit: r1819858 - /apr/apr/trunk/poll/unix/port.c

[Rainer Jung]

Am 02.01.2018 um 19:11 schrieb Eric Covener:

On Tue, Jan 2, 2018 at 12:51 PM, Eric Covener  wrote:

On Tue, Jan 2, 2018 at 12:37 PM, Yann Ylavic  wrote:

On Tue, Jan 2, 2018 at 6:28 PM,   wrote:

Author: ylavic
Date: Tue Jan  2 17:28:53 2018
New Revision: 1819858

URL: http://svn.apache.org/viewvc?rev=1819858&view=rev
Log:
poll, port: re-add the wakeup pipe to the pollset after it triggered.

Just like user fds (file, socket), otherwise it's one shot only (PR-61786).


I can't really compile/test this (no Solaris at hand), reasoning based
on code inspection...



Corresponding test committed in r1819857.


May someone with the {hard,soft}ware test at r1819857 and then at
r1819858 (this commit) to see if it makes a difference?
I expect "testpoll" to wait indefinitely before at r1819857, and work
as usual at r1819858...

Thanks!


I am testing this now on Solaris/x64 and will report back. Thanks for
the great work on event!


testpoll hangs at 1819857 and works at 1819861 (latest)


I did a check on Solaris 10 Sparc. Looks good as well:

- r1819856 (without the additional check and without the fix): no hang 
in testpoll


- r1819857 (with the additional check but still without the fix): hangs 
in testpoll in pollset_wakeup, more precisely in the new for loop in 
iteration 2 of 2 and there in the call to apr_pollset_wakeup().


- r1819858 (with the additional check and with the fix): no hang in testpoll

- r1819902 (head of trunk): no hang in testpoll

Thanks for finding and fixing this!

Happy new year,

Rainer

Re: configure bug - openssl

[Rainer Jung]

Am 17.11.2017 um 04:07 schrieb Helmut K. C. Tessarek:

On 2017-11-15 19:12, Rainer Jung wrote:

IMHO there is no clean and easy workaround but for now it should be
acceptable to set your LIBS environment variable to "-ldl" before
running configure.


Yep, the following worked:

LIBS=-ldl ./configure --prefix=/usr/local/apr-util
--with-apr=/usr/local/apr --with-crypto --with-openssl=/usr/local/ssl

see config.log.3


Yep, looks good, thanks for confirming. We'll look into adding 
pkg-config autofoo to hopefully correctly detect what's needed to link 
apu crypto against OpenSSL. We might be able to take it from the mod_ssl 
m4 macros.


Concerning linking against static libs: in a situation where you link 
APU crypto statically against OpenSSl and also mod_ssl, you might not 
get what you expect. Although the files mod_ssl.so and 
apr_crypto_openssl.so contain independent copies of the needed OpenSSL 
implementation functions, the runtime linker by default would not 
necessarily use them. By default the runtime linker on most Unix/Linux 
platforms looks for symbols in all loaded objects in the order of 
loading. For instance in the hppd case it would first look there, then 
it would search all the objects directly loaded by httpd etc.


For OpenSSL a not-so-nice example would be, that httpd for some reason 
loads ldap system libraries, which themselves are linked against the 
platform OpenSSL, so these are loaded early and searched before mod_ssl 
or apru-til crypto (at least f mod_ssl and apr-util themselves are 
shared objects, not statically linked into httpd). To let the runtime 
linker first search for a symbol definition in the local object, I 
typically add -Bsymbolic (or -Wl,-Bsymbolic for gcc).


Regards,

Rainer

Re: configure bug - openssl

[Rainer Jung]

Too late to write error free mails:

Am 16.11.2017 um 01:12 schrieb Rainer Jung:
On Linux you can use "objdump" to look at dependencies of libraries, so 
for instance


objdump /usr/local/lib/libcrypto.so


should have been:

objdump -p /usr/local/lib/libcrypto.so

By the way: most people use "ldd", but ldd follows the whole treee of 
dependencies. With objdump you can see more unfiltered, what you library 
really wants.


Regards,

Rainer

Re: configure bug - openssl

[Rainer Jung]

Mini post scriptum: I meant "dl" library, not "ld" library (flag -ldl).

Re: configure bug - openssl

[Rainer Jung]

Am 15.11.2017 um 22:02 schrieb Helmut K. C. Tessarek:

On 2017-11-15 03:53, Rainer Jung wrote:

Could you please provide the config.log that you get from that variant?
We should be able from that to see where and why configure fails.


I have attached both config.log files in my first original post:

./configure --prefix=/usr/local/apr-util --with-apr=/usr/local/apr
--with-crypto --with-openssl=/usr/local/ssl

is in config.log.1

./configure --prefix=/usr/local/apr-util --with-apr=/usr/local/apr
--with-crypto --with-openssl=/usr/local/ssl/lib

is in config.log.2


I'd say it is a bug in our configure script. Your OpenSSL crypto lib has 
a dependency on the "dl" library (dynamical loading) which is normal. 
When configure tries to compile a test program to detect whether OpenSSL 
support EVP_CIPHER_CTX_new, linking that program fails because of 
undefined symbols which would be available in the "dl" library:


/usr/local/ssl/lib/libcrypto.a(dso_dlfcn.o): In function 
`dlfcn_globallookup':

dso_dlfcn.c:(.text+0x11): undefined reference to `dlopen'
dso_dlfcn.c:(.text+0x24): undefined reference to `dlsym'
dso_dlfcn.c:(.text+0x2f): undefined reference to `dlclose'
/usr/local/ssl/lib/libcrypto.a(dso_dlfcn.o): In function `dlfcn_bind_func':
dso_dlfcn.c:(.text+0x354): undefined reference to `dlsym'
dso_dlfcn.c:(.text+0x412): undefined reference to `dlerror'
/usr/local/ssl/lib/libcrypto.a(dso_dlfcn.o): In function `dlfcn_bind_var':
dso_dlfcn.c:(.text+0x484): undefined reference to `dlsym'
dso_dlfcn.c:(.text+0x542): undefined reference to `dlerror'
/usr/local/ssl/lib/libcrypto.a(dso_dlfcn.o): In function `dlfcn_load':
dso_dlfcn.c:(.text+0x5a9): undefined reference to `dlopen'
dso_dlfcn.c:(.text+0x60d): undefined reference to `dlclose'
dso_dlfcn.c:(.text+0x645): undefined reference to `dlerror'
/usr/local/ssl/lib/libcrypto.a(dso_dlfcn.o): In function `dlfcn_pathbyaddr':
dso_dlfcn.c:(.text+0x6d1): undefined reference to `dladdr'
dso_dlfcn.c:(.text+0x731): undefined reference to `dlerror'
/usr/local/ssl/lib/libcrypto.a(dso_dlfcn.o): In function `dlfcn_unload':
dso_dlfcn.c:(.text+0x792): undefined reference to `dlclose'
collect2: error: ld returned 1 exit status

IMHO there is no clean and easy workaround but for now it should be 
acceptable to set your LIBS environment variable to "-ldl" before 
running configure. Please don' give up if this still fails, it can well 
be that we see other OpenSSL check failures for other reasons later 
during configure. Simply provide the config.log again and I will have 
another look.


During build time there is a clean way of supplying additional OpenSSL 
dependencies and flags like "-L..." and "-R..." or "-Wl,-rpath,..." 
namely by providing them in the environment variable 
LDADD_crypto_openssl. But this variable is not used during configure time.


Some platforms like Solaris need additional dependency libs, namely 
"-lsocket -lnsl". Some builds of OpenSSL depend on "-lz" for compresion, 
but for safety reasons more modern builds should have compression 
support turned of in OpenSSL.


On your second run, when using the "wrong" path to OpenSSL, the above 
test (for example) ends up doing


configure:4952: checking for EVP_CIPHER_CTX_new in -lcrypto
configure:4977: gcc -o conftest -g -O2 -pthread  -DLINUX -D_REENTRANT 
-D_GNU_SOURCE -I/usr/local/ssl/lib/include -L/usr/local/ssl/lib/lib 
conftest.c -lcrypto   >&5

configure:4977: $? = 0

but since there is not OpenSSL crypto library at /usr/local/ssl/lib/lib 
the program will simply be linked with the platform OpenSSL library and 
that one seems to not have the "dl" dependency.


On Linux you can use "objdump" to look at dependencies of libraries, so 
for instance


objdump /usr/local/lib/libcrypto.so

should show you an extry "NEEDED" for libdl.so.SOMEVERSION

Running the same command for your platform libcrypto.so, that will 
likely not be there. There's nothing wrong with your private OpenSSL 
copy, it just explains why the second configure run showed the differing 
behavior.


As a real fix IMHO we need to add PKG_CONFIG checks to our OpenSSL 
handling in configure, so that we learn from the pc file the OpenSSL 
dependencies. Such code is already present for the nss crypto handling 
in the same configure and is missing from the OpenSSL part.


Regards,

Rainer

Re: configure bug - openssl

[Rainer Jung]

Am 15.11.2017 um 07:55 schrieb Helmut K. C. Tessarek:

On 2017-11-15 01:17, William A Rowe Jr wrote:

There is NO way for autocrud to know the messes in your build environment.

One thought, although we go to effort to strip away other includes and
libs during the additive phase, it is possible your prior package --with
tag helped resolve openssl's path.


If apr-util's configure does not find my openssl which is installed in
/usr/local/ssl with the following configure line:

./configure --prefix=/usr/local/apr-util --with-apr=/usr/local/apr
--with-crypto --with-openssl=/usr/local/ssl


Could you please provide the config.log that you get from that variant? 
We should be able from that to see where and why configure fails.



but with this one:

./configure --prefix=/usr/local/apr-util --with-apr=/usr/local/apr
--with-crypto --with-openssl=/usr/local/ssl/lib

then it is not my build system which is a mess.


Regards,

Rainer

Re: [Announce] Apache Portable Runtime APR 1.6.3, APR-util 1.6.1 and APR-iconv 1.2.2 Released

[Rainer Jung]

Am 01.11.2017 um 12:22 schrieb Stefan Sperling:

On Mon, Oct 23, 2017 at 01:27:59PM -0500, William A Rowe Jr wrote:

   CVE-2017-12618; Out-of-bounds access in corrupted SDBM database.

   APR-util 1.6.0 and prior failed to validate the integrity of SDBM
   database files used by apr_sdbm*() functions, resulting in a
   possible out of bound read access. A local user with write access
   to the database can make a program or process using these functions
   crash, and cause a denial of service.


I am looking for the patch which fixed the above issue.

Where can I find it?

Was it r1809394? All of it? Some of it?

Rationale: APR-util 1.6.3 added a shared library symbol:

No dynamic export changes
PLT added:
 apr_xml_parser_done

I want to figure out a way to patch this security issue in
OpenBSD 6.2-stable, without changing unrelated library symbols.


Yes, it should be r1809394 or even better the 1.6.x backport r1809395.

Regards,

Rainer

Re: svn commit: r1683521 - /apr/apr/trunk/network_io/unix/sockaddr.c

[Rainer Jung]

Am 25.10.2017 um 15:21 schrieb Joe Orton:

On Wed, Oct 25, 2017 at 10:22:46AM +0200, Rainer Jung wrote:

Before the change, copying starts at "scope_delim + 1", after the change at
"scope_delim". The log says "Simplify to use apr_pstrmemdup, no functional
change.", so it seems that was not an intentional change. Shouldn't we
correct it with the following change:


How annoying, sorry :( Yes you're obviously right.


NP, thanks a bunch for your prompt reaction!

Rainer

Re: svn commit: r1683521 - /apr/apr/trunk/network_io/unix/sockaddr.c

[Rainer Jung]

The following comment also applies to the backports to 1.7.x (1808043) 
and 1.6.x (1808043):


Am 04.06.2015 um 13:24 schrieb jor...@apache.org:

Author: jorton
Date: Thu Jun  4 11:24:20 2015
New Revision: 1683521

URL: http://svn.apache.org/r1683521
Log:
* network_io/unix/sockaddr.c (apr_parse_addr_port): Simplify to use
   apr_pstrmemdup, no functional change.

Modified:
 apr/apr/trunk/network_io/unix/sockaddr.c

Modified: apr/apr/trunk/network_io/unix/sockaddr.c
URL: 
http://svn.apache.org/viewvc/apr/apr/trunk/network_io/unix/sockaddr.c?rev=1683521&r1=1683520&r2=1683521&view=diff
==
--- apr/apr/trunk/network_io/unix/sockaddr.c (original)
+++ apr/apr/trunk/network_io/unix/sockaddr.c Thu Jun  4 11:24:20 2015
@@ -277,19 +277,13 @@ APR_DECLARE(apr_status_t) apr_parse_addr
  return APR_EINVAL;
  }
  addrlen = scope_delim - str - 1;
-*scope_id = apr_palloc(p, end_bracket - scope_delim);
-memcpy(*scope_id, scope_delim + 1, end_bracket - scope_delim - 1);
-(*scope_id)[end_bracket - scope_delim - 1] = '\0';
+*scope_id = apr_pstrmemdup(p, scope_delim, end_bracket - 
scope_delim - 1);


Before the change, copying starts at "scope_delim + 1", after the change 
at "scope_delim". The log says "Simplify to use apr_pstrmemdup, no 
functional change.", so it seems that was not an intentional change. 
Shouldn't we correct it with the following change:


Index: network_io/unix/sockaddr.c
===
--- network_io/unix/sockaddr.c  (revision 1809650)
+++ network_io/unix/sockaddr.c  (working copy)
@@ -277,7 +277,7 @@
 return APR_EINVAL;
 }
 addrlen = scope_delim - str - 1;
-*scope_id = apr_pstrmemdup(p, scope_delim, end_bracket - 
scope_delim - 1);
+*scope_id = apr_pstrmemdup(p, scope_delim + 1, end_bracket 
- scope_delim - 1);

 }
 else {
 addrlen = addrlen - 2; /* minus 2 for '[' and ']' */

Regards,

Rainer

Re: [Vote] apr-1.6.3, apr-util-1.6.1, apr-iconv-1.2.2 releases

[Rainer Jung]

Am 18.10.2017 um 16:58 schrieb William A Rowe Jr:

Please cast your votes on the following candidate packages;

Release apr-1.6.3
   [XX] +1 looks good
   [  ] +/-0 since
   [  ] -1 because

Release apr-util-1.6.1
   [XX] +1 looks good
   [  ] +/-0 since
   [  ] -1 because

Release apr-iconv-1.2.2
   [XX] +1 looks good
   [  ] +/-0 since
   [  ] -1 because


General comments:

- Announcement and changes download file not yet present for check.
- you might want to add your (sub?) key to the KEYS file

Detailed APR test results:

- files signed, checksums correct

- svn compared with gz, bz2 and zip only minor differences
  (libtool m4 files in gz and bz2, which seem to not get
   cleaned up by buildconf)

- I built and made check on the following platforms:
  - Solaris 8 Sparc, gcc 4.1.2
  - Solaris 10 Sparc, gcc 7.1.0
  - SuSE Linux Enterprise 10 32, 10, 11 and 12 64 Bit
  - RedHat Enterprise Linux 6 and 7 64 Bit

- config.guess timestamp='2017-09-16',
  pretty much up-to-date

- config.sub timestamp='2017-09-16'
  pretty much up-to-date

- all builds succeeded

- all "make check" ran fine, except for
  - binding to ::1 on Solaris with only IPv4 active,
not a regression:
  testsockets: Line 131: Could not bind socket (126):
  Cannot assign requested address
  Line 189: Condition is false, but expected true
  FAILED 1 of 7
  - Solaris 8 (not a regression):
  testsock: Line 116: Problem generating sockaddr (670008):
  host/servname not known
  Segmentation Fault (coredump)

- Some warnings during "make check":
  - Solaris 8+10 (not a regression)
  testsock: Line 433: Cannot test if connect completes synchronously
  SUCCESS

  - Solaris 8+10, OK doesn't know how to cork
  testsockopt: Line 84: TCP isn't corkable
  SUCCESS

  - Solaris 8, OK doesn't have "unsetenv"
  testenv: Line 75: apr_env_delete
   Line 106: apr_env (skip recycle test_emptyenv)
  SUCCESS

  - Solaris 8, OK doesn't support pollcb
  testpoll: Line 584: pollcb interface not supported
Line 611: pollcb interface not supported
Line 637: pollcb interface not supported
Line 653: pollcb interface not supported
Line 836: pollcb interface not supported
Line 733: pollcb interface not supported
  SUCCESS

  - Linux only 64 Bits: OK, no LFS needed on 64 Bit OS
  testlfs: Line 349: LFS support a no-op in 64-bit builds
  SUCCESS

- tests succeed for apr-util 1.6.1 on top of apr 1.6.3
  for all of the above platforms (all observed test failures
  are old and not related to the new version).


Detailed APR-UTIL test results:

- svn compared with gz, bz2 and zip only expected differences

- files signed, checksums correct

- I built and made check on the following platforms:
  - Solaris 8 Sparc, gcc 4.1.2
  - Solaris 10 Sparc, gcc 7.1.0
  - SuSE Linux Enterprise 10 32, 10, 11 and 12 64 Bit
  - RedHat Enterprise Linux 6 and 7 64 Bit
- using all combinations of:
   - apr 1.6.3
   - expat 2.2.4
   - OpenSSL 1.0.2l (plus some patches)
   - dso disable / enable
   - Berkeley DB 6.1.19
   - sqlite 3.20.1
   - mysql 6.1.10
   - oracle 11.2.0.2.0 (Solaris 10), resp. 10.2.0.5.0 (Solaris 8)
   - platform nss (Solaris 10 and RHEL 6)

- make check for all builds was successful


Detailed APR-ICONV test results:

- svn compared with gz, bz2 and zip only expected differences
  - but autom4te.cache could probably be removed before packaging
tar.gz and tar.bz2

- files signed, checksums correct

- builds fine in combination with APR 1.6.3
  - some compiler warnings

- no check/test make target

I did not actually try to use the build result.


Additional test info:

- httpd test framework looks good for httpd 2.4.29
  using apr/apu 1.6.3/1.6.1 on Solaris 10, SLES 11+12 and RHEL 6+7
  (reallyall module set, shared and static linking,
   MPMs prefork, worker and event, log level trace8)
  I need to give it a final look, details will follow on dev@httpd.

Thanks!

Rainer

Re: AC_CHECK_LIB issues under maintainer mode (Was: Re: Tagging 2.4.29 / 2.5.0-{alpha/beta?} today)

[Rainer Jung]

Am 16.10.2017 um 12:31 schrieb Joe Orton:

On Fri, Oct 13, 2017 at 11:51:54AM -0400, Jim Jagielski wrote:

The long and short is that under maintainer mode, we cannot
expect AC_CHECK_LIB to being correct any longer, because
the combination of -Werror and -Wstrict-prototypes means
that any and all functions looked for/checked for using
AC_CHECK_LIB will NOT be found, due to warnings which are
now fatal errors during configure time, even if those
functions DO exist.


IMO the correct fix is to add all -W... flags to NOTEST_CFLAGS not
CFLAGS so they don't take effect during the configure run at all.  At
least I can't think of a good motivation for having compiler warnings
enabled when running autoconf tests in general.


Good hint, never used that variable. So what about the following patch 
instead: just tried it on trunk and seemed to work fine there. Ut 
changes APACHE_ADD_GCC_CFLAG to operate on NOTEST_CFLAGS instead of 
CFLAGS (the macro currently is only used in places where we IMHO 
actually want that change) and introduces NOTEST_CFLAGS use to configure 
where we handle maintainer mode and debugger mode.


Index: acinclude.m4
===
--- acinclude.m4(revision 1812263)
+++ acinclude.m4(working copy)
@@ -960,7 +960,7 @@
 dnl
 dnl APACHE_ADD_GCC_CFLAGS
 dnl
-dnl Check if compiler is gcc and supports flag. If yes, add to CFLAGS.
+dnl Check if compiler is gcc and supports flag. If yes, add to 
NOTEST_CFLAGS.

 dnl
 AC_DEFUN([APACHE_ADD_GCC_CFLAG], [
   define([ap_gcc_ckvar], [ac_cv_gcc_]translit($1, [-:.=], []))
@@ -973,7 +973,7 @@
   CFLAGS="$save_CFLAGS"
 ])
 if test "$]ap_gcc_ckvar[" = "yes" ; then
-   APR_ADDTO(CFLAGS,[$1])
+   APR_ADDTO(NOTEST_CFLAGS,[$1])
 fi
   fi
   undefine([ap_gcc_ckvar])
Index: configure.in
===
--- configure.in(revision 1812263)
+++ configure.in(working copy)
@@ -627,14 +627,10 @@

AC_ARG_ENABLE(maintainer-mode,APACHE_HELP_STRING(--enable-maintainer-mode,Turn 
on debugging and compile time warnings and load all compiled modules),

 [
   if test "$enableval" = "yes"; then
-APR_ADDTO(CPPFLAGS, -DAP_DEBUG)
+APR_ADDTO(NOTEST_CPPFLAGS, -DAP_DEBUG)
 if test "$GCC" = "yes"; then
-  APR_ADDTO(CFLAGS,[-Wall -Wmissing-prototypes -Wstrict-prototypes 
-Wmissing-declarations -Wpointer-arith])

-  # Next flag needed, because -Wstrict-prototypes in combination with
-  # -Werror leads to compiler errors during configure checks (autoconf
-  # generates incomplete prototypes).
-  APACHE_ADD_GCC_CFLAG([-Wno-error=strict-prototypes])
-  APACHE_ADD_GCC_CFLAG([-std=c89])
+  APR_ADDTO(NOTEST_CFLAGS,[-Wall -Wmissing-prototypes 
-Wstrict-prototypes -Wmissing-declarations -Wpointer-arith])

+  #APACHE_ADD_GCC_CFLAG([-std=c89])
   APACHE_ADD_GCC_CFLAG([-Werror])
   APACHE_ADD_GCC_CFLAG([-Wdeclaration-after-statement])
   APACHE_ADD_GCC_CFLAG([-Wformat])
@@ -641,7 +637,7 @@
   APACHE_ADD_GCC_CFLAG([-Wformat-security])
   APACHE_ADD_GCC_CFLAG([-Wunused])
 elif test "$AIX_XLC" = "yes"; then
-  APR_ADDTO(CFLAGS,-qfullpath -qinitauto=FE -qcheck=all -qinfo=pro)
+  APR_ADDTO(NOTEST_CFLAGS,-qfullpath -qinitauto=FE -qcheck=all 
-qinfo=pro)

 fi
 if test "x$enable_load_all_modules" = "x"; then
   LOAD_ALL_MODULES=yes
@@ -657,9 +653,9 @@

AC_ARG_ENABLE(debugger-mode,APACHE_HELP_STRING(--enable-debugger-mode,Turn 
on debugging and compile time warnings and turn off optimization),

 [
   if test "$enableval" = "yes"; then
-APR_ADDTO(CPPFLAGS, -DAP_DEBUG)
+APR_ADDTO(NOTEST_CPPFLAGS, -DAP_DEBUG)
 if test "$GCC" = "yes"; then
-  APR_ADDTO(CFLAGS,[-Wall -Wmissing-prototypes -Wstrict-prototypes 
-Wmissing-declarations -Wpointer-arith -O0])
+  APR_ADDTO(NOTEST_CFLAGS,[-Wall -Wmissing-prototypes 
-Wstrict-prototypes -Wmissing-declarations -Wpointer-arith -O0])

   APACHE_ADD_GCC_CFLAG([-Wdeclaration-after-statement])
   APACHE_ADD_GCC_CFLAG([-Werror=declaration-after-statement])
   APACHE_ADD_GCC_CFLAG([-Wformat])
@@ -666,7 +662,7 @@
   APACHE_ADD_GCC_CFLAG([-Wformat-security])
   APACHE_ADD_GCC_CFLAG([-Werror=format-security])
 elif test "$AIX_XLC" = "yes"; then
-  APR_ADDTO(CFLAGS,-qfullpath -qinitauto=FE -qcheck=all -qinfo=pro)
+  APR_ADDTO(NOTEST_CFLAGS,-qfullpath -qinitauto=FE -qcheck=all 
-qinfo=pro)

 fi
   fi
 ])dnl


Regards,

Rainer

Re: AC_CHECK_LIB issues under maintainer mode (Was: Re: Tagging 2.4.29 / 2.5.0-{alpha/beta?} today)

[Rainer Jung]

Am 15.10.2017 um 16:25 schrieb Yann Ylavic:

On Sun, Oct 15, 2017 at 4:03 PM, Rainer Jung  wrote:


Why is this happening now? The "-Werror" was backported last December in
r1772330, which was a backport of r1702948 from trunk (May 2015). Maybe
people haven't used maintainer mode since then?


During the backport process of r1772330, Jacob noticed that -Werror
was not working as expected (see STATUS changes in this commit). He
also made a comment on dev@ here:
https://marc.info/?l=apache-cvs&m=147508169616462&w=2

Maybe -Werror is just ignored somehow, because I always compile in
maintainer mode with several gcc versions...


Thanks Yann, I actually only ran gcc with the respective flags. But 
indeed configure checks for each flag whether it is "working" and the 
program which gets compiled is:


int
main ()
{
struct tm tm; tm.tm_gmtoff;
  ;
  return 0;
}

So since we set -Wstrict-prototypes before, -Werror turns this into

conftest.c:45:1: error: function declaration isn't a prototype 
[-Werror=strict-prototypes]

 main ()
 ^~~~

and -Werror does not get set at all.

Nevertheless I would still say that adding 
"-Wno-error=strict-prototypes" for any clang and gcc version that 
supports it would be the correct option. Then -Werror should 
automatically get applied again.


So something like the following (simple) patch should be an improvement 
for gcc and clang and also fix Jim's problem. Of course since we then 
would have -Werror enabled probably for the first time for gcc other new 
problems might show (that will currently only be observable as warnings 
during maintainer builds).


Index: configure.in
===
--- configure.in(revision 1812218)
+++ configure.in(working copy)
@@ -597,6 +597,7 @@
 if test "$GCC" = "yes"; then
   APR_ADDTO(CFLAGS,[-Wall -Wmissing-prototypes -Wstrict-prototypes 
-Wmissing-declarations -Wpointer-arith])

   APACHE_ADD_GCC_CFLAG([-std=c89])
+  APACHE_ADD_GCC_CFLAG([-Wno-error=strict-prototypes])
   APACHE_ADD_GCC_CFLAG([-Werror])
   APACHE_ADD_GCC_CFLAG([-Wdeclaration-after-statement])
   APACHE_ADD_GCC_CFLAG([-Wformat])

Regards,

Rainer

Re: AC_CHECK_LIB issues under maintainer mode (Was: Re: Tagging 2.4.29 / 2.5.0-{alpha/beta?} today)

[Rainer Jung]

Hi Jim,

Am 13.10.2017 um 17:51 schrieb Jim Jagielski:

Let's recall what is really happening...

In maintainer mode, the build system sets -Werror and -Wstrict-prototypes.
This means that functions which lack strict prototypes will "fail".

Now note that AC_CHECK_LIB does not worry about generating
function calls w/ prototypes, so, for example, when checking
for luaL_newstate, it will fail *even if the function exists*!
In fact, this is how I 1st observed the issue: mod_lua was
no longer being included.

The long and short is that under maintainer mode, we cannot
expect AC_CHECK_LIB to being correct any longer, because
the combination of -Werror and -Wstrict-prototypes means
that any and all functions looked for/checked for using
AC_CHECK_LIB will NOT be found, due to warnings which are
now fatal errors during configure time, even if those
functions DO exist.

PS: CCing dev@apr since APR also uses AC_CHECK_LIB


I has a look at this. autoconf does generate a line they call a 
prototype, but it is not a strict prototype (there's no type information 
for the function arguments):


/* Override any GCC internal prototype to avoid an error.
   Use char because int might match the return type of a GCC
   builtin and then its argument prototype would still apply.  */
#ifdef __cplusplus
extern "C"
#endif
char luaL_newstate ();
int
main ()
{
return luaL_newstate ();
  ;
  return 0;
}

Directly before the "int main" there's the broken prototype line

char luaL_newstate ();

And in fact the compiler would also complain about

int
main ()

due to the missing "void" for the arguments.

Of all the flags we set in maintainer mode, the combination that lets 
the compiler fail is:


-Wstrict-prototypes
-Werror

It fails for me also when using GCC (from 4.1.2 to 7.1.0)! So I think a 
clang-specific solution is incomplete.


Why is this happening now? The "-Werror" was backported last December in 
r1772330, which was a backport of r1702948 from trunk (May 2015). Maybe 
people haven't used maintainer mode since then?


Regards,

Rainer

Re: 1.6.0 release candidates

[Rainer Jung]

Am 25.04.2017 um 17:27 schrieb Yann Ylavic:

On Tue, Apr 25, 2017 at 9:55 AM, Yann Ylavic  wrote:


The attached patch prevents this by looping on
pthread_cond_[timed]wait() until the condition is satisfied.
Would you please try it?


Committed to trunk (only) in r1792622, for easier/wider testing possibly.


I applied r1792621, r1792622 and r1792625 and tested on Solaris 8 and 
10. For the test I tried with and without "--enable-timedlocks". I also 
added the hard disabling of (the broken) pthread_mutex_timedlock on 
Solaris 10 before testing.


The tests do no longer hang. The only strangeness is without 
"--enable-timedlocks" on Solaris 10 I get some "error" lines in the "APR 
Lock Performance Test" section (testlockperf.c) for the tests of type 
"(TIMED)":


...
apr_thread_mutex_t Tests
Initializing the apr_thread_mutex_t (TIMED) OK
Starting 2 threadsOK
microseconds: 697841 usec
error: counter = 1837161
...
apr_thread_mutex_t Tests
Initializing the apr_thread_mutex_t (TIMED) OK
Starting 3 threadsOK
microseconds: 1194129 usec
error: counter = 2562724
...
apr_thread_mutex_t Tests
Initializing the apr_thread_mutex_t (TIMED) OK
Starting 4 threadsOK
microseconds: 1890774 usec
error: counter = 3462599
...
apr_thread_mutex_t Tests
Initializing the apr_thread_mutex_t (TIMED) OK
Starting 5 threadsOK
microseconds: 3713934 usec
error: counter = 4897388
...
apr_thread_mutex_t Tests
Initializing the apr_thread_mutex_t (TIMED) OK
Starting 6 threadsOK
microseconds: 2638046 usec
error: counter = 4479066
...

So these tests end too early. The counter should be 100*#threads, 
but it is always smaller.


I do not see such "error" lines for Linux.

Regards,

Rainer

Re: 1.6.0 release candidates

[Rainer Jung]

Hello Yann,

Am 18.04.2017 um 23:37 schrieb Yann Ylavic:

On Tue, Apr 18, 2017 at 4:25 PM, Rainer Jung  wrote:

Am 18.04.2017 um 00:16 schrieb Yann Ylavic:


All the above hopefuly fixes with r1791718 and r1791728, both
backported to 1.6.x.


Sorry to say that, but there's now another strange test result on Solaris 8.
Again testprocmutex. It fails or mech "default_timed" in test_exclusive
here:

   209  rv = apr_proc_mutex_unlock(proc_lock);
   210  APR_ASSERT_SUCCESS(tc, "unlock after timedlock check", rv);
   211
   212  *x = 0;
   213
   214  for (n = 0; n < CHILDREN; n++)
   215  make_child(tc, -1, &child[n], p);
   216
   217  for (n = 0; n < CHILDREN; n++)
   218  await_child(tc, child[n]);
^^^

In await_child while waiting for the number 0 child, apr_proc_wait returns
after 75 seconds with code 1.

When I try t use truss, I had to terminate it after a few minutes, because
it wasn't terminating by itself. I had about half 550.000 lwp_mutex_lock and
590.000 lwp_mutex_unlock for each of the the 6 threads,


Do you really mean more unlocks than locks?
I don't see how locks/unlocks could be unbalanced in
proc_mutex_pthread_acquire_ex() and proc_mutex_pthread_release().

For each the testprocmutex's child process loop, there should be a max
of 2 locks+unlocks per proc_mutex_pthread_acquire_ex() (given that
pthread_cond_timedwait() itself releases and re-acquires the mutex if
it's called), and 1 lock+unlock per proc_mutex_pthread_release().
That's something like 3 x CHILDREN x MAX_ITER = 3600 locks and as much
unlocks for the whole "default_timed" test, if no timeout occurs!
According to your numbers, that'd mean ~150 timeouts/retries on a
total of 1200 loops (~12.5%).


and also about
40.000 per thread of:

lwp_cond_wait(0xFF040018, 0xFF04, 0xFFBEF868) Err#62 ETIME


So < 10% of the 2400 proc_mutex_pthread_acquire_ex()'s locks+unlocks
come from pthread_cond_timedwait(), which means most of the
apr_proc_mutex_timedlock() end up being simple {pthread_mutex_lock();
locked=1; pthread_mutex_unlock()}, and sounds to me like a costly
mutex implementation on Solaris 8...



Any idea what to look for in the condvar impl of the timedlock feature?


Since I wrote it I may be missing something, but I see nothing wrong yet...
We could optimize it for the non contended case by using atomics on
the shared proc_pthread_mutex_t->cond_locked state still.

How is the following patch working, by waiting for a more or less
longer time on the mutex in the test (here 10ms)?

Index: test/testprocmutex.c
===
--- test/testprocmutex.c(revision 1791728)
+++ test/testprocmutex.c(working copy)
@@ -87,11 +87,12 @@ static void make_child(abts_case *tc, int trylock,
 }
 else if (trylock < 0) {
 int wait_usec = 0;
+apr_interval_time_t timeout = apr_time_from_msec(10);

-while ((rv = apr_proc_mutex_timedlock(proc_lock, 1))) {
+while ((rv = apr_proc_mutex_timedlock(proc_lock, timeout))) {
 if (!APR_STATUS_IS_TIMEUP(rv))
 exit(1);
-if (++wait_usec >= MAX_WAIT_USEC)
+if (++wait_usec >= MAX_WAIT_USEC / timeout)
 exit(1);
 }
 }
_



Thanks for pointing me at that code. I tried a variation of timeouts. 
The test on Solaris 10 nearly always fails here *after some time*.


The wait_usec loop tying the apr_proc_mutex_timedlock() very often 
succeeds already during the first or second, sometimes the third 
attempt. But then the outer loop "while (i < MAX_ITER)" suddenly starts 
to fail because the inner loop switches behavior and 
apr_proc_mutex_timedlock() always gets the timeout until we stop trying 
due to wait_usec. Increasing MAX_WAIT_USEC from 1 sec to 5 sec didn't 
help. I have no idea, what the root cause for this behavior switching is.


I also tried Solaris specific pthread_cond_reltimedwait_np() instead of 
pthread_cond_timedwait() to rule out some time calculation problems and 
deltas between clocks APR might use and the CLOCK that the system impl 
uses. That didn't help though.


It also does not seem to be a problem with spurious wakeups or early 
return from the timed waiting.


Regards,

Rainer

Re: 1.6.0 release candidates

[Rainer Jung]

Am 18.04.2017 um 16:25 schrieb Rainer Jung:

Am 18.04.2017 um 00:16 schrieb Yann Ylavic:

Thanks Rainer for the detailed report (as usual).

On Mon, Apr 17, 2017 at 2:01 PM, Rainer Jung 
wrote:


c) Failure to compile apr on Solaris 8
--

Using gcc 4.1.2 compiling locks/unix/proc_mutex.c:

 In function 'proc_mutex_pthread_create':
607: error: lvalue required as left operand of assignment
 In function 'proc_mutex_pthread_acquire_ex':
711: error: lvalue required as left operand of assignment
790: warning: implicit declaration of function 'pthread_mutex_timedlock'
 In function 'proc_mutex_pthread_release':
868: error: lvalue required as left operand of assignment
 In function 'proc_mutex_pthread_cond_create':
945: error: lvalue required as left operand of assignment

Concerning pthread_mutex_timedlock: that platform has
HAVE_PTHREAD_CONDATTR_SETPSHARED set to 1 but
HAVE_PTHREAD_MUTEX_TIMEDLOCK
is not defined (and pthread_mutex_timedlock not available). So
APR_USE_PROC_PTHREAD_MUTEX_COND is defined. So in line 688 we enter the
"#else" branch which ends in line 815. That code includes a call to
pthread_mutex_timedlock.

Concerning the "lvalue required as left operand of assignment" the
lines are
always of the form

proc_pthread_mutex_cond_locked(SOME_MUTEX) = SOME_INT;

and proc_pthread_mutex_cond_locked(m) is defined as

((m)->pthread_refcounting ? proc_pthread_cast(m)->cond_locked : -1)

which indeed doesn't look like a good lvalue.


Sorry about that, I thought I had tested the
APR_USE_PROC_PTHREAD_MUTEX_COND case with some #undef's on Linux but
it seems I missed it for my latest changes.

All the above hopefuly fixes with r1791718 and r1791728, both
backported to 1.6.x.


Sorry to say that, but there's now another strange test result on
Solaris 8. Again testprocmutex. It fails or mech "default_timed" in
test_exclusive here:

   209  rv = apr_proc_mutex_unlock(proc_lock);
   210  APR_ASSERT_SUCCESS(tc, "unlock after timedlock check", rv);
   211
   212  *x = 0;
   213
   214  for (n = 0; n < CHILDREN; n++)
   215  make_child(tc, -1, &child[n], p);
   216
   217  for (n = 0; n < CHILDREN; n++)
   218  await_child(tc, child[n]);
^^^

In await_child while waiting for the number 0 child, apr_proc_wait
returns after 75 seconds with code 1.

When I try t use truss, I had to terminate it after a few minutes,
because it wasn't terminating by itself. I had about half 550.000
lwp_mutex_lock and 590.000 lwp_mutex_unlock for each of the the 6
threads, and also about 40.000 per thread of:

lwp_cond_wait(0xFF040018, 0xFF04, 0xFFBEF868) Err#62 ETIME

Any idea what to look for in the condvar impl of the timedlock feature?


After disabling pthread_mutex_timedlock for Solaris 10, I get the same 
problem as described above. I also tried with Linux and had mixed 
results. One attempt on SLES 11 did not show the problem, another 
attempt with RHEL 7 did show the same problem.


The duration for which the await_process hangs varies, I saw 164 seconds 
on Solaris10 and 234 seconds for RHEL 7.


Regards,

Rainer

Re: 1.6.0 release candidates

[Rainer Jung]

Am 18.04.2017 um 00:16 schrieb Yann Ylavic:

Thanks Rainer for the detailed report (as usual).

On Mon, Apr 17, 2017 at 2:01 PM, Rainer Jung  wrote:


c) Failure to compile apr on Solaris 8
--

Using gcc 4.1.2 compiling locks/unix/proc_mutex.c:

 In function 'proc_mutex_pthread_create':
607: error: lvalue required as left operand of assignment
 In function 'proc_mutex_pthread_acquire_ex':
711: error: lvalue required as left operand of assignment
790: warning: implicit declaration of function 'pthread_mutex_timedlock'
 In function 'proc_mutex_pthread_release':
868: error: lvalue required as left operand of assignment
 In function 'proc_mutex_pthread_cond_create':
945: error: lvalue required as left operand of assignment

Concerning pthread_mutex_timedlock: that platform has
HAVE_PTHREAD_CONDATTR_SETPSHARED set to 1 but HAVE_PTHREAD_MUTEX_TIMEDLOCK
is not defined (and pthread_mutex_timedlock not available). So
APR_USE_PROC_PTHREAD_MUTEX_COND is defined. So in line 688 we enter the
"#else" branch which ends in line 815. That code includes a call to
pthread_mutex_timedlock.

Concerning the "lvalue required as left operand of assignment" the lines are
always of the form

proc_pthread_mutex_cond_locked(SOME_MUTEX) = SOME_INT;

and proc_pthread_mutex_cond_locked(m) is defined as

((m)->pthread_refcounting ? proc_pthread_cast(m)->cond_locked : -1)

which indeed doesn't look like a good lvalue.


Sorry about that, I thought I had tested the
APR_USE_PROC_PTHREAD_MUTEX_COND case with some #undef's on Linux but
it seems I missed it for my latest changes.

All the above hopefuly fixes with r1791718 and r1791728, both
backported to 1.6.x.


Sorry to say that, but there's now another strange test result on 
Solaris 8. Again testprocmutex. It fails or mech "default_timed" in 
test_exclusive here:


   209  rv = apr_proc_mutex_unlock(proc_lock);
   210  APR_ASSERT_SUCCESS(tc, "unlock after timedlock check", rv);
   211
   212  *x = 0;
   213
   214  for (n = 0; n < CHILDREN; n++)
   215  make_child(tc, -1, &child[n], p);
   216
   217  for (n = 0; n < CHILDREN; n++)
   218  await_child(tc, child[n]);
^^^

In await_child while waiting for the number 0 child, apr_proc_wait 
returns after 75 seconds with code 1.


When I try t use truss, I had to terminate it after a few minutes, 
because it wasn't terminating by itself. I had about half 550.000 
lwp_mutex_lock and 590.000 lwp_mutex_unlock for each of the the 6 
threads, and also about 40.000 per thread of:


lwp_cond_wait(0xFF040018, 0xFF04, 0xFFBEF868) Err#62 ETIME

Any idea what to look for in the condvar impl of the timedlock feature?

Regards,

Rainer

Re: 1.6.0 release candidates

[Rainer Jung]

Am 18.04.2017 um 00:16 schrieb Yann Ylavic:

Thanks Rainer for the detailed report (as usual).

On Mon, Apr 17, 2017 at 2:01 PM, Rainer Jung  wrote:


c) Failure to compile apr on Solaris 8
--

Using gcc 4.1.2 compiling locks/unix/proc_mutex.c:

 In function 'proc_mutex_pthread_create':
607: error: lvalue required as left operand of assignment
 In function 'proc_mutex_pthread_acquire_ex':
711: error: lvalue required as left operand of assignment
790: warning: implicit declaration of function 'pthread_mutex_timedlock'
 In function 'proc_mutex_pthread_release':
868: error: lvalue required as left operand of assignment
 In function 'proc_mutex_pthread_cond_create':
945: error: lvalue required as left operand of assignment

Concerning pthread_mutex_timedlock: that platform has
HAVE_PTHREAD_CONDATTR_SETPSHARED set to 1 but HAVE_PTHREAD_MUTEX_TIMEDLOCK
is not defined (and pthread_mutex_timedlock not available). So
APR_USE_PROC_PTHREAD_MUTEX_COND is defined. So in line 688 we enter the
"#else" branch which ends in line 815. That code includes a call to
pthread_mutex_timedlock.

Concerning the "lvalue required as left operand of assignment" the lines are
always of the form

proc_pthread_mutex_cond_locked(SOME_MUTEX) = SOME_INT;

and proc_pthread_mutex_cond_locked(m) is defined as

((m)->pthread_refcounting ? proc_pthread_cast(m)->cond_locked : -1)

which indeed doesn't look like a good lvalue.


Sorry about that, I thought I had tested the
APR_USE_PROC_PTHREAD_MUTEX_COND case with some #undef's on Linux but
it seems I missed it for my latest changes.

All the above hopefuly fixes with r1791718 and r1791728, both
backported to 1.6.x.


The lvalue problem is gone in 1.6.x head, I can compile also on Solaris 
8 (and still on my other test platforms).


Running make check, I get a core dump in testsock.c. In test_addr_copy 
the initial


rv = apr_sockaddr_info_get(&sa1, *host, APR_UNSPEC, 80, 0, p);
APR_ASSERT_SUCCESS(tc, "Problem generating sockaddr", rv);

fails when host == "::1". In the following

rv = apr_sockaddr_info_copy(&sa2, sa1, p);
APR_ASSERT_SUCCESS(tc, "Problem copying sockaddr", rv);

sa1 is NULL and the copy results in a segmentation fault.

The copy functionality and the test is new in 1.6.x.

IMHO this is more a test problem and not an impl problem, sa1 being not 
NULL is probably an expected prerequisite for the copy call.


I'll try to add some proper skip to the test if the 
apr_sockaddr_info_get() fails.


I haven't yet tracked down, why this call fails for Solaris 8, but I am 
not too concerned about IPv6 and ancient Solaris 8.


Regards,

Rainer

Re: 1.6.0 release candidates

[Rainer Jung]

Am 18.04.2017 um 12:09 schrieb Nick Kew:

On Mon, 2017-04-17 at 17:06 +0100, Nick Kew wrote:

   And I need to do some more digging
around that bogus PGP key!


OK, this follows a subject that's been raised @apache before:
https://mail-search.apache.org/members/private-arch/members/201606.mbox/%3c1464999260.7490.275.ca...@mimir.webthing.com%3E
following which apache's own pages were fixed to stop using
32-bit key IDs.

Underlying story is at https://evil32.com/ .  I think I shall also
blog this story and add my own thoughts.


Thank a bunch. So I had imported the wrong key resp. the right and the 
wrong key by only using the short form of the fingerprint.


It turns out, that for my keys also invalid clones with the same short 
fingerprint exist :(


I will be more careful in the future, using the full fingerprint.

Thanks again!

Rainer

Re: 1.6.0 release candidates

[Rainer Jung]

Am 18.04.2017 um 00:16 schrieb Yann Ylavic:

d) Hang during APR make check on Solaris 10 (testprocmutex)
---

pthread_mutex_timedlock() hangs when the current thread already has locked
the mutex.

...

Assuming the Solaris code is similar to:

https://github.com/OpenIndiana/illumos-gate/blame/master/usr/src/lib/libc/port/threads/synch.c

maybe my Solaris 10 does not have the following code change in it:

https://github.com/OpenIndiana/illumos-gate/commit/f52756fb59521fc0f684db03ee24da2a1d12a52a

"6738798 pthread_mutex_timedlock can block forever when using priority
inherit mutexes"

When running the same testall binary on Solaris 11, I do not get these
hangs.

Unfortunately I was not (yet) able to find out, whether there's a patch for
Bug 6738798 available on Solaris 10, or whether we would break Solaris 10.


Maybe we need to "#define APR_USE_PROC_PTHREAD_MUTEX_COND 0" for
Solaris 10, and fall back to the generic implementation (spinning
sleep)...


Oracle changed the bug id from 6738798 to 15502514. The bug isn't rated 
high and the fix never got released for Solaris 10, so all Solaris 10 
systems should be buggy.


The fix was included in Solaris 11, probably from the beginning, so 
those systems should be OK.


So we either need to allow to disable the feature, or find a fix for it 
(which IMHO is hard, because the hang happens before we can detect that 
we will hang) or we disable the mutex_timedlock feature automatically 
for Solaris 10. Based on the above status for Solaris 10, I am for 
automatic disabling on Solaris 10 during configure time.


Regards,

Rainer

Re: 1.6.0 release candidates

[Rainer Jung]

Hi Nick,

Am 17.04.2017 um 18:06 schrieb Nick Kew:

On Mon, 2017-04-17 at 14:01 +0200, Rainer Jung wrote:

Thanks for the comments.


my test results: most important is failure to compile on Solaris 8 and a
hang during make test on Solaris 10, as well as your key seeming to be
revoked. See below.

1) Formal observations
==

- MD5 and SHA1 missing


Indeed.  Do you think those should be encouraged, as they don't
actually give any more security than a simple checksum?


I thought those are not for security but to easily detect transfer 
errors. Only the pgp signature is for security. The hashes still seem to 
be frequently provided.



- zip archives not named "win32-src.zip" as previously
- CHANGES-APR-1.6 and CHANGES-APR-UTIL-1.6 missing



- HEADER.html and README.html missing
- Announcement1.6.txt and Announcement1.6.html missing


AFAICS those don't feature in the tarballs?  I looked at
the 1.5.latest of APR/APU and I don't see them.


Right, those files will not be part of the artefacts but exist in the 
download directory, e.g.:


https://dist.apache.org/repos/dist/release/apr/

or

http://archive.apache.org/dist/apr/


- Key B87F79A9 missing from KEYS file


Good point.  Since ASF now auto-generates per-project keys files
daily, why maintain a separate KEYS file?


Didn't know about that feature. Can it also be used to check historic 
signatures, e.g. of old releases? At least we would have to rephrase


http://apr.apache.org/download.cgi

the KEYS file is mentioned there.


- Key B87F79A9 is listed as "revoked: 2016-08-16" in key server


I saw that key after uploading the tarballs!
That key has a wrong creation date and wrong fingerprint:
it's an imposter who created a clash (I recollect discussing
that, though I only just found out that a fake with my name
on it was "out there")!  My real key is in
http://people.apache.org/keys/group/apr.asc and
http://people.apache.org/keys/committer/
(along with my old 1024-bit key, which I can also use if this
is creating confusion).


Thanks for correcting this.


- ".svn" directory included, that's unusual
- STATUS file included, that's unusual


It's a fair cop, guv!


- Unix build files configure, *.spec, build-outputs.mk, build/*.sh,
   build/l*.m4 (libtool m4 files) and the copies of the apr build/*.m4
   in apu are included in zip (that's new, but IMHO OK)


Ah, right.  Yep, that's probably not such a good idea.

Do we in fact still need those in the Unix tarballs?  Are there
platforms out there with the toolchain for configure/make but not
for buildconf?


I always thought about deleting the build/l*.m4 files coming from 
libtool while running buildconf at the end of buildconf. I think they 
are not needed for building and if you rerun buildoncf they will again 
get copied from the libtool installation.


IMHO we should stick to configure/make as the normal build procedure and 
not demand people to run buildconf. Users who build should not have a 
need to care about libtool and auto tools installations. So I'd keep the 
files generated by buildconf for the Unix tarballs (except for the 
build/l*.m4 files as mentioned above).



- no more bundled expat: we need to note that in the release notes


Agreed, that was certainly on the agenda for the announcement.



2) Compile and run
==

a) New apr configure options
b) Changes in apr-util configure options


Useful for release notes :)


c) Failure to compile apr on Solaris 8
--


Damn.  That looks like a showstopper for release.  But shouldn't
be too hard, since you've tracked down the cause.


Yes I think that one should get fixed.


which indeed doesn't look like a good lvalue.


Indeedie.


d) Hang during APR make check on Solaris 10 (testprocmutex)
---

pthread_mutex_timedlock() hangs when the
 current thread already has
locked the mutex.


Hmmm, OK.  We knew the timedlock stuff had potential to blow up.
I guess that points back to the plan of making it a config option.


I'll try to dig some more info whether the problem can be fixed by an OS 
patch for Solaris 10. At least it shows the potential for problems.


I'm a bit undecided on the config option, because that means we'll have 
builds with subtle API differences. Most of the other APR config options 
are not a good comparison for a consistent decision.



I'm out this evening, but will hopefully find time tomorrow to
revisit these problems.  And I need to do some more digging
around that bogus PGP key!


Thanks!

Rainer

Re: 1.6.0 release candidates

[Rainer Jung]

Hi Nick,

Am 16.04.2017 um 01:12 schrieb Nick Kew:

Today I have tagged both APR and APR-UTIL 1.6.0
and rolled tarballs of release candidates in
the full choice of formats provided for 1.5.latest
on our pages.

Please download and test, from
http://people.apache.org/~niq/apr/


my test results: most important is failure to compile on Solaris 8 and a 
hang during make test on Solaris 10, as well as your key seeming to be 
revoked. See below.


1) Formal observations
==

- MD5 and SHA1 missing
- zip archives not named "win32-src.zip" as previously
- CHANGES-APR-1.6 and CHANGES-APR-UTIL-1.6 missing
- HEADER.html and README.html missing
- Announcement1.6.txt and Announcement1.6.html missing
- Key B87F79A9 missing from KEYS file
- Key B87F79A9 is listed as "revoked: 2016-08-16" in key server
- ".svn" directory included, that's unusual
- STATUS file included, that's unusual
- Unix build files configure, *.spec, build-outputs.mk, build/*.sh,
  build/l*.m4 (libtool m4 files) and the copies of the apr build/*.m4
  in apu are included in zip (that's new, but IMHO OK)
- no more bundled expat: we need to note that in the release notes

2) Compile and run
==

a) New apr configure options


  --enable-allocator-guard-pages  Use guard pages in apr_allocator
  (implies --enable-allocator-uses-mmap)
  --enable-pool-concurrency-check Check for concurrent usage of memory 
pools

  --with-valgrind[=DIR]   Enable code to teach valgrind about apr pools
  (optionally: set path to valgrind headers)


b) Changes in apr-util configure options


  - new --with-commoncrypto=DIR specify location of CommonCrypto
  - removed --with-freetds=DIR  specify FreeTDS location
  - changed --with-expat=DIR from "specify Expat location, or 
'builtin'" to "specify Expat location"



c) Failure to compile apr on Solaris 8
--

Using gcc 4.1.2 compiling locks/unix/proc_mutex.c:

 In function 'proc_mutex_pthread_create':
607: error: lvalue required as left operand of assignment
 In function 'proc_mutex_pthread_acquire_ex':
711: error: lvalue required as left operand of assignment
790: warning: implicit declaration of function 'pthread_mutex_timedlock'
 In function 'proc_mutex_pthread_release':
868: error: lvalue required as left operand of assignment
 In function 'proc_mutex_pthread_cond_create':
945: error: lvalue required as left operand of assignment

Concerning pthread_mutex_timedlock: that platform has 
HAVE_PTHREAD_CONDATTR_SETPSHARED set to 1 but 
HAVE_PTHREAD_MUTEX_TIMEDLOCK is not defined (and pthread_mutex_timedlock 
not available). So APR_USE_PROC_PTHREAD_MUTEX_COND is defined. So in 
line 688 we enter the "#else" branch which ends in line 815. That code 
includes a call to pthread_mutex_timedlock.


Concerning the "lvalue required as left operand of assignment" the lines 
are always of the form


proc_pthread_mutex_cond_locked(SOME_MUTEX) = SOME_INT;

and proc_pthread_mutex_cond_locked(m) is defined as

((m)->pthread_refcounting ? proc_pthread_cast(m)->cond_locked : -1)

which indeed doesn't look like a good lvalue.


d) Hang during APR make check on Solaris 10 (testprocmutex)
---

pthread_mutex_timedlock() hangs when the current thread already has 
locked the mutex.


GDB says:

#0  0xff14ca00 in ___lwp_mutex_timedlock () from /lib/libc.so.1
No symbol table info available.
#1  0xff13fde8 in mutex_lock_kernel () from /lib/libc.so.1
No symbol table info available.
#2  0xff140f4c in stall () from /lib/libc.so.1
No symbol table info available.
#3  0xff1416dc in mutex_lock_internal () from /lib/libc.so.1
No symbol table info available.
#4  0xff141cc0 in pthread_mutex_timedlock () from /lib/libc.so.1
No symbol table info available.
#5  0xff369310 in proc_mutex_pthread_acquire_ex (mutex=0xb4ff8, 
timeout=1492421091353146)

at .../locks/unix/proc_mutex.c:790
abstime = {tv_sec = 1492421091, tv_nsec = 353146000}
rv = 
#6  0xff36a500 in apr_proc_mutex_timedlock (mutex=0xb4ff8, timeout=1) at 
.../locks/unix/proc_mutex.c:1558

No locals.
#7  0x00029f90 in test_exclusive (lockname=0x0, mech=0xffbffad8, 
tc=0xffbffa70) at .../test/testprocmutex.c:197

child = {0xb5118, 0xb5128, 0xb5138, 0xb5148, 0xb5158, 0xb5168}
rv = 0
n = 0
#8  proc_mutex (tc=0xffbffa70, data=0xffbffad8) at 
.../test/testprocmutex.c:243

rv = 
shmname = 0x37ad0 "tpm.shm"
shm = 0xb4fc8
#9  0x00015a4c in abts_run_test (ts=ts@entry=0xb7b70, f=f@entry=0x29ccc 
, value=value@entry=0xffbffad8)

at .../test/abts.c:171
tc = {failed = 0, suite = 0xb76a8}
ss = 0xb76a8
#10 0x0002a26c in testprocmutex (suite=0xb7b70) at 
.../test/testprocmutex.c:274
lockmechs = {{num = APR_LOCK_DEFAULT, name = 0x37d70 
"default"}, {num = APR_LOCK_SYSVSEM, name = 0x37

Re: No timed lock on MacOS

[Rainer Jung]

Am 05.04.2017 um 16:25 schrieb Branko Čibej:

On 05.04.2017 14:16, Jim Jagielski wrote:

Hmmm Looking over the timed stuff, it seems that semtimedop()
is used incorrectly.

For both pthread_mutex_timedlock() and sem_timedwait(), the
timeout variable is the actual wallclock time that the wait
expires (eg: now+5mins). But, from what I can read about semtimedop(), its
timeout really is a timeout (5mins). But it seems that we use it
as if it was wall-clock time, which is WAY wrong...

Can someone check to see what exactly the timeout value is for
semtimedop()? Is it a delta or a wall-clock?



My man page says:

semtimedop() behaves identically to semop() except that in those
cases where the calling thread would sleep, the duration of that
sleep is limited by the amount of elapsed time specified by the
timespec structure whose address is passed in the timeout argument.


Looks like delta to me.


Solaris man page. Those are typically good:

 The semtimedop() function behaves as semop() except when  it
 must  suspend  execution  of the calling process to complete
 its operation.  If semtimedop()  must  suspend  the  calling
 process   after  the  time  interval  specified  in  timeout
   ^^
 expires, or if the timeout  expires  while  the  process  is
 suspended,  semtimedop()  returns  with  an  error.  If  the
 timespec structure pointed to by timeout is zero-valued  and
 semtimedop()  needs  to  suspend the calling process to com-
 plete the requested  operation(s),  it  returns  immediately
 with  an error. If timeout is the NULL pointer, the behavior
 of semtimedop() is identical to that of semop().

So they speak of a timeout interval, which also makes "delta" the likely 
interpretation.


Rainer

Re: Default Linux mutex method

[Rainer Jung]

Am 01.04.2017 um 19:02 schrieb Yann Ylavic:

On Fri, Mar 31, 2017 at 4:58 PM, Nick Kew  wrote:



On 31 Mar 2017, at 03:21, William A Rowe Jr  wrote:

So almost two decades later, this is still odd.


Is this a reference to some technical discussion on the subject you recollect?

Have you verified that all APR_USE_*_SERIALIZE are mutually exclusive,
or could it be that we use both in different contexts, perhaps for a valid 
reason?


apr.h:#define APR_USE_SHMEM_MMAP_TMP 0
apr.h:#define APR_USE_SHMEM_MMAP_SHM 0
apr.h:#define APR_USE_SHMEM_MMAP_ZERO0
apr.h:#define APR_USE_SHMEM_SHMGET_ANON  0
apr.h:#define APR_USE_SHMEM_SHMGET   1
apr.h:#define APR_USE_SHMEM_MMAP_ANON1
apr.h:#define APR_USE_SHMEM_BEOS 0
apr.h:#define APR_USE_FLOCK_SERIALIZE   0
apr.h:#define APR_USE_SYSVSEM_SERIALIZE 1
apr.h:#define APR_USE_POSIXSEM_SERIALIZE0
apr.h:#define APR_USE_FCNTL_SERIALIZE   0
apr.h:#define APR_USE_PROC_PTHREAD_SERIALIZE0
apr.h:#define APR_USE_PTHREAD_SERIALIZE 1


I’ve just (finally) got the toolchain to build & test this on MacOS,
and I’ve got identical APR_USE results there.  Perhaps our build logic
really isn’t doing anything very smart at all?


Looks good to me (as said in the other message), nothing mutually
exclusive AFAICT.




SYSVSEM > PTHREAD in the code logic, so we are still using
sysv semaphores with all their defects over the choice of pthread
mutexes where both are supported on Linux.


The main defect being the leak of an (limited) IPC entry when httpd
does not shutdown cleanly, and that it's slower than pthreads, though
it's hardly measurable probably.



Probably applies more widely than just Linux.  But might a general fix
risk breaking some minority Unix platform or cross-compile?


SYSVSEM are robust (can be recovered after a process owning one
crashes), so the switch to PTHREAD should depend on
HAVE_PTHREAD_MUTEX_ROBUST only I think (which should be the case for
Linux and Solaris at least).

Solaris had issue a while ago ([1], which seems to be the "why" for
prefering SYSVSEM over some shared-pthread implementations), but it's
probably addressed now (Rainer maybe?).


First current autodetect results for Solaris 8 (ancient) and 10 (still 
in use). I don't have 11 results easily at hand but could provide them:


Solaris 8

#define APR_USE_SHMEM_MMAP_TMP 0
#define APR_USE_SHMEM_MMAP_SHM 0
#define APR_USE_SHMEM_MMAP_ZERO0
#define APR_USE_SHMEM_SHMGET_ANON  0
#define APR_USE_SHMEM_SHMGET   1
#define APR_USE_SHMEM_MMAP_ANON1
#define APR_USE_SHMEM_BEOS 0
#define APR_USE_FLOCK_SERIALIZE   0
#define APR_USE_SYSVSEM_SERIALIZE 0
#define APR_USE_POSIXSEM_SERIALIZE0
#define APR_USE_FCNTL_SERIALIZE   1
#define APR_USE_PROC_PTHREAD_SERIALIZE0
#define APR_USE_PTHREAD_SERIALIZE 1

Solaris 10:

the same except for

#define APR_USE_FCNTL_SERIALIZE   0 (instead of 1)
#define APR_USE_PROC_PTHREAD_SERIALIZE1 (instead of 0)

Both versions detect/set HAVE_PTHREAD_MUTEX_ROBUST during configure.

I don't have first hand knowledge, but the topic came up a couple of 
times. There's this old conversation between Joe and Jeff:


http://marc.info/?t=10872098193&r=1&w=2

Then we have included 5 years later since APR 1.3.10:

http://svn.apache.org/viewvc?rev=881301&view=rev

From the log:

Solaris 10 or later: Use pthread by default for cross-process mutex 
instead of fcntl; the latter is not thread-aware, leading to EDEADLK 
failures with some multi-threaded, multi-process applications.


Proper recovery after a crash of the mutex owner was tested on Solaris 
10 U5 and OpenSolaris 2009.06. Earlier Solaris versions weren't tested, 
and there were reports in httpd-land long ago about failures to recover 
the mutex.



The list discussion (triggered by Jim originally not restricted to 
Solaris) that lead to this decision can be found here:


http://marc.info/?t=12384365733&r=1&w=2

It contains some more details.

I do not know anything more recent then contained there.


Do we want to fix this in the 1.6.0 release?


How confident are you of not breaking anything with a fix?


The only proc_pthread issue I can think of is [2] (long thread), which
we addressed in both 1.5.x and 1.6.x, so +1 for me for the switch
(when HAVE_PTHREAD_MUTEX_ROBUST).


Regards,
Yann.

[1] https://bz.apache.org/bugzilla/show_bug.cgi?id=21322#c18
[2] https://bz.apache.org/bugzilla/show_bug.cgi?id=49504


Regards,

Rainer

Re: testpoll

[Rainer Jung]

Am 10.01.2017 um 16:20 schrieb Rainer Jung:

Am 10.01.2017 um 15:35 schrieb Jim Jagielski:

Occasionally, I get:

  testpoll: /Line 816: expected <0>, but saw <4>

Anyway I could easily see what test, exactly, is failing?
./testall -v testpoll does nil.


Depending on which version of APR you are testing, it should be line 816
of file test/testpoll.c, e.g. in 1.5.x and 1.6.x HEAD:

...
815 rv = apr_pollset_poll(pollset, -1, &num, &descriptors);
816 ABTS_INT_EQUAL(tc, APR_SUCCESS, rv);
817 ABTS_INT_EQUAL(tc, 1, num);
...

checking the return code of the apr_pollset_poll() call in 815 against
APR_SUCCESS. So I guess the expected <0> is APR_SUCCESS and the real
return code was 4.


OK, at least on RHEL 6 errno 4 is EINTR.

Regards,

Rainer

Re: testpoll

[Rainer Jung]

Am 10.01.2017 um 16:20 schrieb Rainer Jung:

Am 10.01.2017 um 15:35 schrieb Jim Jagielski:

Occasionally, I get:

  testpoll: /Line 816: expected <0>, but saw <4>

Anyway I could easily see what test, exactly, is failing?
./testall -v testpoll does nil.


Depending on which version of APR you are testing, it should be line 816
of file test/testpoll.c, e.g. in 1.5.x and 1.6.x HEAD:

...
815 rv = apr_pollset_poll(pollset, -1, &num, &descriptors);
816 ABTS_INT_EQUAL(tc, APR_SUCCESS, rv);
817 ABTS_INT_EQUAL(tc, 1, num);
...

checking the return code of the apr_pollset_poll() call in 815 against
APR_SUCCESS. So I guess the expected <0> is APR_SUCCESS and the real
return code was 4.


But maybe your line numbers in testpoll.c differ, because return value 4 
seems to not make much sense.


Regards,

Rainer

Re: testpoll

[Rainer Jung]

Am 10.01.2017 um 15:35 schrieb Jim Jagielski:

Occasionally, I get:

  testpoll: /Line 816: expected <0>, but saw <4>

Anyway I could easily see what test, exactly, is failing?
./testall -v testpoll does nil.


Depending on which version of APR you are testing, it should be line 816 
of file test/testpoll.c, e.g. in 1.5.x and 1.6.x HEAD:


...
815 rv = apr_pollset_poll(pollset, -1, &num, &descriptors);
816 ABTS_INT_EQUAL(tc, APR_SUCCESS, rv);
817 ABTS_INT_EQUAL(tc, 1, num);
...

checking the return code of the apr_pollset_poll() call in 815 against 
APR_SUCCESS. So I guess the expected <0> is APR_SUCCESS and the real 
return code was 4.


Regards,

Rainer

Re: [Bug 45615] "Large Files not supported" with 64-bit build

[Rainer Jung]

Hi Dennis,

Am 10.01.2017 um 13:36 schrieb Dennis Clarke:


re: https://bz.apache.org/bugzilla/show_bug.cgi?id=45615

I think it best to follow up here as per suggestions by Yann and Rainer
wherein I can run further tests and experiments to determine what is
happening here in these Niagara class systems.

Firstly, sorry for awaking what seems like a long dead cold bug but it
really isn't a "Large Files not supported" bug as opposed to just a
message that needs to be tweaked.  Indeed yes this is a 64 bit build and
so off_t and size_t and going to be 64-bit sized :


I will have a look at the message "Line 345: Large Files not supported" 
coming from "make check" and see whether I can tweak it ("not supported 
or not needed" or similar). Due to the API used by APR I think we can't 
be more precise (effectively distinguish between the two cases) easily.


...


As for the blocking rand, well hrmmm, not unless I was trying to read a
ton of data from /dev/random wherein :

Devicesrandom(7D)

NAME
 random, urandom - Strong random number generator device

SYNOPSIS
 /dev/random
 /dev/urandom

DESCRIPTION
 The /dev/random and /dev/urandom  files  are  special  files
 that  are  a source for random bytes generated by the kernel
 random  number  generator  device.   The   /dev/random   and
 /dev/urandom  files  are suitable for applications requiring
 high quality random numbers for cryptographic purposes.

 The generator device produces random numbers from  data  and
 devices available to the kernel and estimates  the amount of
 randomness (or entropy) collected from  these  sources.  The
 entropy  level  determines the amount of high quality random
 numbers that are produced at a given time.

 Applications retrieve random bytes by reading /dev/random or
 /dev/urandom. The /dev/random interface returns random bytes
 only when sufficient amount of entropy has  been  collected.
 If  there  is  no entropy to produce the requested number of
 bytes,  /dev/random  blocks  until  more  entropy   can   be
 obtained.  Non-blocking  I/O mode can be used to disable the
 blocking behavior. The /dev/random interface  also  supports
 poll(2). Note that using poll(2) will not increase the speed
 at which random numbers can be read.


etc etc ...

So I would expect that /dev/random may slow down but not by too much
while many other processes are running on the system and in this case
the build system is a single virtual zone inside a machine with a
number of zones.  There should be plenty of data. However I have never
run a benchmark. I can certainly run a test for "blocking rand()" as
per http://pubs.opengroup.org/onlinepubs/9699919799/functions/rand.html


However I will say that the Apache 2.4.25 server seems to be running
very well with this new apr and apr-util but I am sure we can sort out
this weird test behavior. I certainly have the Sparc hardware sitting
here and can even provide the Oracle Sparc M7 tests if needed.


/dev/random can block because many server systems quickly run out of 
entropy.


If you could follow my suggestions for further debugging your hanging or 
extremely long running apr "make check" (using pstack, prstat, truss as 
written in the ticket), I'm sure we can find the culprit. If it is 
/dev/random, we can find that in this type of debug info, if it is 
something else likely we can also determine it from the three.


Regards,

Rainer

Re: svn commit: r1733684 - in /apr/apr/branches/1.6.x: ./ include/ include/arch/netware/ include/arch/os2/ include/arch/unix/ locks/beos/ locks/netware/ locks/os2/ locks/unix/ locks/win32/ test/

[Rainer Jung]

Hi Yann,

Am 19.12.2016 um 01:55 schrieb Yann Ylavic:

On Sat, Dec 17, 2016 at 6:48 PM, Rainer Jung  wrote:

This commit introduced a new dependency on pthread_mutex_timedlock. Although
its presence is detected by configure and locks/unix/thread_mutex.c checks
HAVE_PTHREAD_MUTEX_TIMEDLOCK and provides an alternative implementation if
it is not defined, file locks/unix/proc_mutex.c uses pthread_mutex_timedlock
unconditionally.


Thanks Rainer for noticing, hopefully fixed in r1774973.


looks good from visually checking.


I wonder if we could fall back to using pthread_cond_timewait() like
we do for thread_mutex, but that'd require
pthread_condattr_setpshared() to be available on the system.
Is it the case at least on this older Solaris?


Yes, Solaris 8 has it, get and set. They are in the man pages, in 
pthread.h and I find symbols in libpthread.so.


The full pthread function list taken from the Solaris 8 man page:

pthread_atfork()
pthread_attr_destroy()
pthread_attr_getdetachstate()
pthread_attr_getguardsize()
pthread_attr_getinheritsched()
pthread_attr_getschedparam()
pthread_attr_getschedpolicy()
pthread_attr_getscope()
pthread_attr_getstackaddr()
pthread_attr_getstacksize()
pthread_attr_init()
pthread_attr_setdetachstate()
pthread_attr_setguardsize()
pthread_attr_setinheritsched()
pthread_attr_setschedparam()
pthread_attr_setschedpolicy()
pthread_attr_setscope()
pthread_attr_setstackaddr()
pthread_attr_setstacksize()
pthread_cancel()
pthread_cleanup_pop()
pthread_cleanup_push()
pthread_cond_broadcast()
pthread_cond_destroy()
pthread_cond_init()
pthread_cond_signal()
pthread_cond_timedwait()
pthread_cond_wait()
pthread_condattr_destroy()
pthread_condattr_getpshared()
pthread_condattr_init()
pthread_condattr_setpshared()
pthread_create()
pthread_detach()
pthread_equal()
pthread_exit()
pthread_getconcurrency()
pthread_getschedparam()
pthread_getspecific()
pthread_join()
pthread_key_create()
pthread_key_delete()
pthread_kill()
pthread_mutex_destroy()
pthread_mutex_getprioceiling()
pthread_mutex_init()
pthread_mutex_lock()
pthread_mutex_setprioceiling()
pthread_mutex_trylock()
pthread_mutex_unlock()
pthread_mutexattr_destroy()
pthread_mutexattr_getprioceiling()
pthread_mutexattr_getprotocol()
pthread_mutexattr_getpshared()
pthread_mutexattr_gettype()
pthread_mutexattr_init()
pthread_mutexattr_setprioceiling()
pthread_mutexattr_setprotocol()
pthread_mutexattr_setpshared()
pthread_mutexattr_settype()
pthread_once()
pthread_rwlock_destroy()
pthread_rwlock_init()
pthread_rwlock_rdlock()
pthread_rwlock_tryrdlock()
pthread_rwlock_trywrlock()
pthread_rwlock_unlock()
pthread_rwlock_wrlock()
pthread_rwlockattr_destroy()
pthread_rwlockattr_getpshared()
pthread_rwlockattr_init()
pthread_rwlockattr_setpshared()
pthread_self()
pthread_setcancelstate()
pthread_setcanceltype()
pthread_setconcurrency()
pthread_setschedparam()
pthread_setspecific()
pthread_sigmask()
pthread_testcancel()


It's possibly not worth it too, thoughts?


I can't judge, I haven't really looked into what the code is doing and 
how useful it would be. Just noticed the compile problem and hte 
inconsistency between the thread and proc cases. Sorry.


Regards,

Rainer

Re: svn commit: r1733684 - in /apr/apr/branches/1.6.x: ./ include/ include/arch/netware/ include/arch/os2/ include/arch/unix/ locks/beos/ locks/netware/ locks/os2/ locks/unix/ locks/win32/ test/

[Rainer Jung]

This commit introduced a new dependency on pthread_mutex_timedlock. 
Although its presence is detected by configure and 
locks/unix/thread_mutex.c checks HAVE_PTHREAD_MUTEX_TIMEDLOCK and 
provides an alternative implementation if it is not defined, file 
locks/unix/proc_mutex.c uses pthread_mutex_timedlock unconditionally.


One platform that doesn't have it is ancient Solaris 8.

I don't really care whether we require it for APR 1.6 or not, but 
providing an alternative in thread_mutex.c but not in proc_mutex.c seems 
inconsistent.


Sorry for the late feedback, but I just noticed this during testing the 
httpd 2.4.25 release candidate with heads of 1.5.x and 1.6.x.


Regards,

Rainer

Am 05.03.2016 um 02:40 schrieb yla...@apache.org:

Author: ylavic
Date: Sat Mar  5 01:40:58 2016
New Revision: 1733684

URL: http://svn.apache.org/viewvc?rev=1733684&view=rev
Log:
Merge r930508, r1667900, r1667901, r1667903, r1667962, r1669077, r1671292, 
r1732582 from trunk:

OS/2: Add an implementation of condition variables, derived from the Win32
implementation.


locks: introduce apr_{thread,proc,global}_mutex_timedlock().

For proc mutexes, the new mechanism APR_LOCK_DEFAULT_TIMED usable at creation 
time
allows for the best mechanism to be elected (unixes: 1 to 3, or specific: 4 to 
7):
1. PROC_PTHREAD if pthread_mutex_timedlock() and pthread_mutex_set_robust_np()
   are both available,
2. SYSV if semtimedop() is availale,
3. POSIXSEM if sem_timedwait() is available,
4. BeOS' acquire_sem_etc() if available,
5. NetWare falls back to apr_thread_mutex_timedlock() as for others functions,
6. OS2's DosRequestMutexSem(),
7. Windows' WaitForSingleObject().
Otherwise (like when fcntl and flock only are availble, if that's ever 
possible),
APR_ENOTIMPL is returned.

For thread mutexes, the new flag APR_THREAD_MUTEX_TIMED, usable at create()
time still, allows to switch to an implementation using a condition variable
and apr_thread_cond_timedwait() when if no native mechanism is available (eg.
NetWare, pthreads but without pthread_mutex_timedlock() available).
On windows, this initializes a WaitForSingleObject()able handle (Mutex) instead
of the fastest (but not timeout-able) CRITICAL_SECTION used by default.

All apr_{thread,proc,global}_mutex_timedlock() functions can take a relative or
absolute time, thanks to the last (boolean) argument.

Test suite updated accordingly.


Follow up to r1667900: revert spurious change on test/abts_tests.h.


Follow up to r1667900: fix comments.


Follow up to r1667900: handle negative (infinite) timeout in mutex/cond 
timedlock/timedwait.


locks: follow up to r1667900.
In apr_global_mutex_timedlock(), we can avoid converting from relative to
absolute time if thread locking is not needed.


make internal function static to avoid a warning


Follow up to r1667900: Avoid a circular reference (PR 59068).


Submitted by: bjh, ylavic, ylavic, ylavic, ylavic, ylavic, trawick, ylavic
Reviewed/backported by: ylavic

Modified:
apr/apr/branches/1.6.x/   (props changed)
apr/apr/branches/1.6.x/configure.in
apr/apr/branches/1.6.x/include/apr_global_mutex.h
apr/apr/branches/1.6.x/include/apr_proc_mutex.h
apr/apr/branches/1.6.x/include/apr_thread_mutex.h
apr/apr/branches/1.6.x/include/apr_time.h
apr/apr/branches/1.6.x/include/arch/netware/apr_arch_thread_mutex.h
apr/apr/branches/1.6.x/include/arch/os2/apr_arch_thread_cond.h
apr/apr/branches/1.6.x/include/arch/unix/apr_arch_proc_mutex.h
apr/apr/branches/1.6.x/include/arch/unix/apr_arch_thread_mutex.h
apr/apr/branches/1.6.x/locks/beos/proc_mutex.c
apr/apr/branches/1.6.x/locks/beos/thread_cond.c
apr/apr/branches/1.6.x/locks/beos/thread_mutex.c
apr/apr/branches/1.6.x/locks/netware/proc_mutex.c
apr/apr/branches/1.6.x/locks/netware/thread_cond.c
apr/apr/branches/1.6.x/locks/netware/thread_mutex.c
apr/apr/branches/1.6.x/locks/os2/proc_mutex.c
apr/apr/branches/1.6.x/locks/os2/thread_cond.c
apr/apr/branches/1.6.x/locks/os2/thread_mutex.c
apr/apr/branches/1.6.x/locks/unix/global_mutex.c
apr/apr/branches/1.6.x/locks/unix/proc_mutex.c
apr/apr/branches/1.6.x/locks/unix/thread_cond.c
apr/apr/branches/1.6.x/locks/unix/thread_mutex.c
apr/apr/branches/1.6.x/locks/win32/proc_mutex.c
apr/apr/branches/1.6.x/locks/win32/thread_cond.c
apr/apr/branches/1.6.x/locks/win32/thread_mutex.c
apr/apr/branches/1.6.x/test/testglobalmutex.c
apr/apr/branches/1.6.x/test/testlock.c
apr/apr/branches/1.6.x/test/testlockperf.c
apr/apr/branches/1.6.x/test/testmutexscope.c
apr/apr/branches/1.6.x/test/testprocmutex.c

Propchange: apr/apr/branches/1.6.x/
--
--- svn:mergeinfo (original)
+++ svn:mergeinfo Sat Mar  5 01:40:58 2016
@@ -1,4 +1,4 @@
 /apr/apr/branches/1.4.x:1003369,1101301
-/apr/apr/trunk:733052,739635,741862,741866-741867,741869,741871,745763-745764,746310,747990,748080,748361,748371,748565

Re: svn commit: r1736521 - in /apr/apr/branches/1.6.x: ./ include/ include/arch/unix/ poll/unix/ test/

[Rainer Jung]

Hi Olli,

Am 10.08.2016 um 13:46 schrieb olli hauer:

On 2016-08-10 12:41, Rainer Jung wrote:

I found some more differences between trunk and r1736521 in 1.6.x which are all 
due to incomplete backports of changes applied to trunk after r899905. I 
backported all of those because they seemed applicable for 1.6.x. At least the 
poll directory, testpoll.c, apr_poll.h and arch/unix/apr_arch_poll_private.h 
are now in sync between trunk and 1.6.x.

Regards,

Rainer



Hi Rainer,

since you are looking at the moment to apr poll, perhaps you can also take a 
look int Bug ID 59914?


I'm sorry: the changes I applied to poll this morning were all of very 
formal nature and simple to verify. I don't know enough about the code 
to efficiently decide on the patch that you have provided for that bug.


Regards,

Rainer

Re: svn commit: r1736521 - in /apr/apr/branches/1.6.x: ./ include/ include/arch/unix/ poll/unix/ test/

[Rainer Jung]

Hi Yann,

Am 10.08.2016 um 12:50 schrieb Yann Ylavic:

Hi Rainer,

On Wed, Aug 10, 2016 at 12:41 PM, Rainer Jung  wrote:

I found some more differences between trunk and r1736521 in 1.6.x which are
all due to incomplete backports of changes applied to trunk after r899905. I
backported all of those because they seemed applicable for 1.6.x. At least
the poll directory, testpoll.c, apr_poll.h and
arch/unix/apr_arch_poll_private.h are now in sync between trunk and 1.6.x.


Is 1.5.x already fine with these backports/fixes?
APR_POLLSET_WAKEABLE is available there too...


I *think* all those changes were followups to the change that introduced 
apr_pollcb_wakeup() and apr_pollcb_method_name() to trunk. That was only 
backported to 1.6, not 1.5. The older apr_pollset_wakeup() and 
apr_pollset_method_name() shouldn't have been touched, so 1.5 should be 
still fine.


Regards,

Rainer

Re: svn commit: r1736521 - in /apr/apr/branches/1.6.x: ./ include/ include/arch/unix/ poll/unix/ test/

[Rainer Jung]

I found some more differences between trunk and r1736521 in 1.6.x which 
are all due to incomplete backports of changes applied to trunk after 
r899905. I backported all of those because they seemed applicable for 
1.6.x. At least the poll directory, testpoll.c, apr_poll.h and 
arch/unix/apr_arch_poll_private.h are now in sync between trunk and 1.6.x.


Regards,

Rainer

Am 09.08.2016 um 23:31 schrieb Rainer Jung:

Am 25.03.2016 um 02:19 schrieb minf...@apache.org:

Author: minfrin
Date: Fri Mar 25 01:19:34 2016
New Revision: 1736521

URL: http://svn.apache.org/viewvc?rev=1736521&view=rev
Log:
Add apr_pollcb_wakeup(), with similar behavior to
apr_pollset_wakeup(). Add apr_pollcb_method_name(), with similar
behavior to apr_pollset_method_name().



...


Modified: apr/apr/branches/1.6.x/poll/unix/port.c
URL:
http://svn.apache.org/viewvc/apr/apr/branches/1.6.x/poll/unix/port.c?rev=1736521&r1=1736520&r2=1736521&view=diff

==

--- apr/apr/branches/1.6.x/poll/unix/port.c (original)
+++ apr/apr/branches/1.6.x/poll/unix/port.c Fri Mar 25 01:19:34 2016
@@ -413,7 +413,7 @@ static apr_status_t impl_pollset_poll(ap
 if ((pollset->flags & APR_POLLSET_WAKEABLE) &&
 fp.desc_type == APR_POLL_FILE &&
 fp.desc.f == pollset->wakeup_pipe[0]) {
-apr_pollset_drain_wakeup_pipe(pollset);
+apr_poll_drain_wakeup_pipe(pollset->wakeup_pipe);
 rv = APR_EINTR;
 }
 else {
@@ -466,9 +466,8 @@ static apr_pollset_provider_t impl = {

 apr_pollset_provider_t *apr_pollset_provider_port = &impl;

-static apr_status_t cb_cleanup(void *p_)
+static apr_status_t impl_pollcb_cleanup(apr_pollcb_t *pollcb)
 {
-apr_pollcb_t *pollcb = (apr_pollcb_t *) p_;
 close(pollcb->fd);
 return APR_SUCCESS;
 }
@@ -505,7 +504,6 @@ static apr_status_t impl_pollcb_create(a
 }

 pollcb->pollset.port = apr_palloc(p, size * sizeof(port_event_t));
-apr_pool_cleanup_register(p, pollcb, cb_cleanup,
apr_pool_cleanup_null);

 return APR_SUCCESS;
 }
@@ -558,16 +556,25 @@ static apr_status_t impl_pollcb_poll(apr
  apr_pollcb_cb_t func,
  void *baton)
 {
-apr_pollfd_t *pollfd;
 apr_status_t rv;
-unsigned int i, nget = 1;
+unsigned int nget = 1;

 rv = call_port_getn(pollcb->fd, pollcb->pollset.port,
pollcb->nalloc,
 &nget, timeout);

 if (nget) {
+unsigned int i;
+
 for (i = 0; i < nget; i++) {
-pollfd = (apr_pollfd_t
*)(pollcb->pollset.port[i].portev_user);
+apr_pollfd_t *pollfd = (apr_pollfd_t
*)(pollcb->pollset.port[i].portev_user);
+
+if ((pollfd->flags & APR_POLLSET_WAKEABLE) &&
+pollfd->desc_type == APR_POLL_FILE &&
+pollfd->desc.f == pollcb->wakeup_pipe[0]) {
+apr_poll_drain_wakeup_pipe(pollcb->wakeup_pipe);
+return APR_EINTR;
+}
+
 pollfd->rtnevents =
get_revent(pollcb->pollset.port[i].portev_events);

 rv = func(baton, pollfd);
@@ -586,6 +593,7 @@ static apr_pollcb_provider_t impl_cb = {
 impl_pollcb_add,
 impl_pollcb_remove,
 impl_pollcb_poll,
+impl_pollcb_cleanup,
 "port"
 };


I get a compilation error (ggc on Solaris):

.../poll/unix/port.c: In function 'impl_pollcb_poll':
.../poll/unix/port.c:571:24: error: 'apr_pollfd_t' has no member named
'flags'
 if ((pollfd->flags & APR_POLLSET_WAKEABLE) &&
^

Maybe the following patch is appropriate?

--- poll/unix/port.c (revision 1755647)
+++ poll/unix/port.c (working copy)
@@ -568,7 +568,7 @@
 for (i = 0; i < nget; i++) {
 apr_pollfd_t *pollfd = (apr_pollfd_t
*)(pollcb->pollset.port[i].portev_user);

-if ((pollfd->flags & APR_POLLSET_WAKEABLE) &&
+if ((pollcb->flags & APR_POLLSET_WAKEABLE) &&
 pollfd->desc_type == APR_POLL_FILE &&
 pollfd->desc.f == pollcb->wakeup_pipe[0]) {
 apr_poll_drain_wakeup_pipe(pollcb->wakeup_pipe);

At least it builds and "make check" runs fine.

Regards,

Rainer

Re: svn commit: r1736521 - in /apr/apr/branches/1.6.x: ./ include/ include/arch/unix/ poll/unix/ test/

[Rainer Jung]

Am 25.03.2016 um 02:19 schrieb minf...@apache.org:

Author: minfrin
Date: Fri Mar 25 01:19:34 2016
New Revision: 1736521

URL: http://svn.apache.org/viewvc?rev=1736521&view=rev
Log:
Add apr_pollcb_wakeup(), with similar behavior to
apr_pollset_wakeup(). Add apr_pollcb_method_name(), with similar
behavior to apr_pollset_method_name().



...


Modified: apr/apr/branches/1.6.x/poll/unix/port.c
URL: 
http://svn.apache.org/viewvc/apr/apr/branches/1.6.x/poll/unix/port.c?rev=1736521&r1=1736520&r2=1736521&view=diff
==
--- apr/apr/branches/1.6.x/poll/unix/port.c (original)
+++ apr/apr/branches/1.6.x/poll/unix/port.c Fri Mar 25 01:19:34 2016
@@ -413,7 +413,7 @@ static apr_status_t impl_pollset_poll(ap
 if ((pollset->flags & APR_POLLSET_WAKEABLE) &&
 fp.desc_type == APR_POLL_FILE &&
 fp.desc.f == pollset->wakeup_pipe[0]) {
-apr_pollset_drain_wakeup_pipe(pollset);
+apr_poll_drain_wakeup_pipe(pollset->wakeup_pipe);
 rv = APR_EINTR;
 }
 else {
@@ -466,9 +466,8 @@ static apr_pollset_provider_t impl = {

 apr_pollset_provider_t *apr_pollset_provider_port = &impl;

-static apr_status_t cb_cleanup(void *p_)
+static apr_status_t impl_pollcb_cleanup(apr_pollcb_t *pollcb)
 {
-apr_pollcb_t *pollcb = (apr_pollcb_t *) p_;
 close(pollcb->fd);
 return APR_SUCCESS;
 }
@@ -505,7 +504,6 @@ static apr_status_t impl_pollcb_create(a
 }

 pollcb->pollset.port = apr_palloc(p, size * sizeof(port_event_t));
-apr_pool_cleanup_register(p, pollcb, cb_cleanup, apr_pool_cleanup_null);

 return APR_SUCCESS;
 }
@@ -558,16 +556,25 @@ static apr_status_t impl_pollcb_poll(apr
  apr_pollcb_cb_t func,
  void *baton)
 {
-apr_pollfd_t *pollfd;
 apr_status_t rv;
-unsigned int i, nget = 1;
+unsigned int nget = 1;

 rv = call_port_getn(pollcb->fd, pollcb->pollset.port, pollcb->nalloc,
 &nget, timeout);

 if (nget) {
+unsigned int i;
+
 for (i = 0; i < nget; i++) {
-pollfd = (apr_pollfd_t *)(pollcb->pollset.port[i].portev_user);
+apr_pollfd_t *pollfd = (apr_pollfd_t 
*)(pollcb->pollset.port[i].portev_user);
+
+if ((pollfd->flags & APR_POLLSET_WAKEABLE) &&
+pollfd->desc_type == APR_POLL_FILE &&
+pollfd->desc.f == pollcb->wakeup_pipe[0]) {
+apr_poll_drain_wakeup_pipe(pollcb->wakeup_pipe);
+return APR_EINTR;
+}
+
 pollfd->rtnevents = 
get_revent(pollcb->pollset.port[i].portev_events);

 rv = func(baton, pollfd);
@@ -586,6 +593,7 @@ static apr_pollcb_provider_t impl_cb = {
 impl_pollcb_add,
 impl_pollcb_remove,
 impl_pollcb_poll,
+impl_pollcb_cleanup,
 "port"
 };


I get a compilation error (ggc on Solaris):

.../poll/unix/port.c: In function 'impl_pollcb_poll':
.../poll/unix/port.c:571:24: error: 'apr_pollfd_t' has no member named 
'flags'

 if ((pollfd->flags & APR_POLLSET_WAKEABLE) &&
^

Maybe the following patch is appropriate?

--- poll/unix/port.c (revision 1755647)
+++ poll/unix/port.c (working copy)
@@ -568,7 +568,7 @@
 for (i = 0; i < nget; i++) {
 apr_pollfd_t *pollfd = (apr_pollfd_t 
*)(pollcb->pollset.port[i].portev_user);


-if ((pollfd->flags & APR_POLLSET_WAKEABLE) &&
+if ((pollcb->flags & APR_POLLSET_WAKEABLE) &&
 pollfd->desc_type == APR_POLL_FILE &&
 pollfd->desc.f == pollcb->wakeup_pipe[0]) {
 apr_poll_drain_wakeup_pipe(pollcb->wakeup_pipe);

At least it builds and "make check" runs fine.

Regards,

Rainer

Re: Apr-Util compatibility with OpenSSL 1.1.0

[Rainer Jung]

Hi Bert,

Am 29.03.2016 um 11:31 schrieb Bert Huijben:

Hi,

I just tried using OpenSSL 1.1.0-pre4 with APR-UTIL and found that the code
is incompatible. The file apr_crypto_openssl.c fails to compile as some
structs have been made opaque.

I can probably provide a patch later, but perhaps others want to know about
the problem before that.


I addressed this in apr trunk and the HEAD of apr-util 1.5.x.

There was no apr-util release since then. If you have the necessary auto 
tool buildchain installed, you can check out apr-util 1.5.x HEAD, run 
"buildconf" against any standard apr and check, whether the result 
builds against OpenSSL 1.1.0-pre4 in your environment (and runs "make 
test" well). I'd be very interested in the feedback!


The changes I applied were:

- r1728969
- r1728971
- r1730342

For APR trunk the changes were:

- r1728958
- r1728963
- r1730344

Regards,

Rainer

Re: Is 1.4.x deprecated? (Was: svn commit: r1728973 - in /apr/apr-util/branches/1.4.x: ./ CHANGES)

[Rainer Jung]

Am 08.02.2016 um 18:53 schrieb William A Rowe Jr:

On Sun, Feb 7, 2016 at 8:47 AM, mailto:rj...@apache.org>> wrote:


Backport of r1728971 from 1.5.x.

Modified:
 apr/apr-util/branches/1.4.x/   (props changed)
 apr/apr-util/branches/1.4.x/CHANGES


Is 1.4.x maintained?

The release of 1.5.x introduced an ABI-compatible replacement of 1.4.x.
The last release of 1.4.x was in March, 2012, with the release of 1.5.0
following on August, 2012.

I'd prefer we treat the 1.4.x and prior branches closed upon the successful
release of the compatible successor to that prior release, as there is no
apparent interest in actually releasing code from the 1.4.x branch.  We
now have 2.x trunk, apr-util 1.5.x head and apr 1.6.x and 1.5.x branches
to maintain as it is.

I last raised this question on 22 Dec '15 in response to
Subject: Re: [PATCH] Prevent huge pool allocations for small requests.
Message-ID:
mailto:ts7uasewccye...@mail.gmail.com>>

but it seems this discussion has not yet picked up.


The same now. I guess no response here means we can deprecate 1.4, 
because there's no real interest in maintaining it.


Since the above change needs a followup change and is not a security 
change, I plan to revert it instead.


Regards,

Rainer

Re: Mailing list semantics

[Rainer Jung]

Am 28.01.2016 um 05:08 schrieb William A Rowe Jr:


[ ] Keep reply-to-poster default reply semantics
[X] Change to reply-to-list default reply semantics


Thanks,

Rainer

Re: Time for APR 2.0?

[Rainer Jung]

Am 28.08.2015 um 15:20 schrieb Michael Schlenker:

Am 28.08.2015 um 01:53 schrieb Noel Butler:




That's why I want us to extend cmake to do the unix build as well.






The biggest bitch I here from *nix/bsd system admins  everywhere is
cmake, the largest bitch is people know how to use --with-foo --with-bar
  now after decades, THEY, the people you expect to use your software,
have to learn it no longer works that way, I know of multinationals
change software because of cmake, I've seen a well known large
international issue a policy that they will remain on old unsupported
versions of mysql because they moved to cmake.

autoconf et al, might be ol school, but every man, woman, and dog knows
and understands it,


Knows yes. Understands autoconf/m4/libtool in depth: Highly in doubt
about that claim, the vast amount of cargo culted libtool/m4/configure
macros floating around speaks against it.

Autoconf/libtool is great when it works. But if it does not, you are in
for a world of pain.

CMake is different yes, and has quite a few warts of its own, but it is
vastly simpler to understand and reason about than libtool/configure/etc.

Regarding cmake as a solution to get rid of those dsp/dsw files:

YES!

Cmake provided project/nmake/make files have one issue though, which
might not be obvious to someone never using cmake: Those depend on cmake
for a variety of tasks (e.g. install tasks are often done by invoking
some 'cmake -E' stuff and so on, via extra build tasks).


I want to strengthen this point: IMHO very similar to configure, using 
cmake as a generator without having cmake on the system where you run 
the build likely will not work. You can't just run it at release time, 
generate nmake or vcxproj files and the consumer only uses MSVC to do 
the build. The end user will need to run cmake on his system. I have 
seen cmake calls in the generated nmake files, but also full absolute 
paths to extracted source files etc. in the generated vcxproj files. So 
those can't be generated at release time.


I'm not terribly experienced with cmake though and maybe it has a more 
friendly generation mode.


Despite those problems, I don't know of a better way to maintain the 
build artefacts we need to ensure windows platform support.



So if one killed off the dsp/dsw files, it gets harder to adjust and
tweak the created projects in VisualStudio, you need to fix the CMake
file and recreate the project file. Which isn't that much different to
the way autoconf works with configure.in etc.

CMake does a few things nicer than libtool/autoconf, especially when you
are NOT a linux distro builder. For example it doesn't force you to use
-rpath everywhere like libtool does, can create installers/rpms etc. out
of the box and has various other conveniences.

But to make it really good, there should be some standard FindAPR.cmake
file provided (and sent to kitware for inclusion), to make it easier for
packages using apr to detect the bits, pieces and parts.


Rainer

Re: GLIBC_2.10 && libapr-1.so

[Rainer Jung]

Hi Matthias,

Am 23.10.2014 um 15:02 schrieb Matthias Apitz:

El día Thursday, October 23, 2014 a las 08:43:22AM +0200, Matthias Apitz 
escribió:


nm .libs/libapr-1.so.0.4.5 | egrep 'GLIBC_2.[4-9]|GLIBC_2.10'
  U accept4@@GLIBC_2.10
  U dup3@@GLIBC_2.9
  U epoll_create1@@GLIBC_2.9

With setting:

apr_cv_accept4=no export apr_cv_accept4
apr_cv_dup3=no export apr_cv_dup3
apr_cv_epoll_create1=no export apr_cv_epoll_create1

we have:

nm .libs/libapr-1.so.0.4.5 | egrep 'GLIBC_2.[4-9]|GLIBC_2.10'
  U pthread_mutex_consistent_np@@GLIBC_2.4
  U pthread_mutexattr_setprotocol@@GLIBC_2.4
  U pthread_mutexattr_setrobust_np@@GLIBC_2.4

Thanks for this hint


Well, it's fine that the symbols for the new calls accept4, dup3,
epoll_create1 went away, so far so good.

But the apache is unwilling to create a LISTEN on its port; it says:


# strace -o /tmp/apa.tr -f /usr/local/sisis-pap/apache/bin/apachectl start
[ Process PID=6864 runs in 32 bit mode. ]
[Thu Oct 23 14:42:12 2014] [crit] (22)Invalid argument: alloc_listener: failed 
to get a socket for (null)
Syntax error on line 138 of /usr/local/sisis-pap/apache/conf/httpd.conf:
Listen setup failed
[ Process PID=6836 runs in 64 bit mode. ]

6864  connect(3, {sa_family=AF_INET6, sin6_port=htons(80), inet_pton(AF_INET6, 
"::", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, 28) = 0
6864  getsockname(3, {sa_family=AF_INET6, sin6_port=htons(32772), inet_pton(AF_INET6, 
"::1", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, [8589934620]) = 0
6864  connect(3, {sa_family=AF_UNSPEC, sa_data="\0\0\0\0\0\0\0\0\0\0\0\0\0\0"}, 
16) = 0
6864  connect(3, {sa_family=AF_INET, sin_port=htons(80), 
sin_addr=inet_addr("0.0.0.0")}, 16) = 0
6864  getsockname(3, {sa_family=AF_INET6, sin6_port=htons(32772), inet_pton(AF_INET6, 
":::127.0.0.1", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, 
[8589934620]) = 0
6864  close(3)  = 0
6864  socket(PF_INET6, 0x80001 /* SOCK_??? */, IPPROTO_IP) = -1 EINVAL (Invalid 
argument)
6864  socket(PF_INET, 0x80001 /* SOCK_??? */, IPPROTO_IP) = -1 EINVAL (Invalid 
argument)
6864  gettimeofday({3233802550310820, 18426497511976861723}, NULL) = 0
6864  open("/etc/localtime", O_RDONLY)  = 3

It seems a known issue, for example:

https://bugzilla.redhat.com/show_bug.cgi?id=516331

The exact version is httpd-2.2.22 which brings APR 1.4.5

Any idea? Thanks


Try to also set

apr_cv_sock_cloexec=no

Again check apr_private.h after configure for having HAVE_SOCK_CLOEXEC 
not set.


Got that additional dependency from

https://issues.apache.org/bugzilla/show_bug.cgi?id=47662

What you try to achieve is not supported, so it is expected to be a bit 
of trial-and-error.


Regards,

Rainer

Re: GLIBC_2.10 && libapr-1.so

[Rainer Jung]

Am 22.10.2014 um 15:13 schrieb Matthias Apitz:

We compile and deliver to our customers in a large Library Project an
Apache and the libapr-1.so, among other stuff.

The compilation is done on our production Linux box:

guru@srap08dxoh:~/apr-1.5.1> uname -a
Linux srap08dxoh 3.0.76-0.11-pae #1 SMP Fri Jun 14 08:21:43 UTC 2013 (ccab990) 
i686 athlon i386 GNU/Linux

with GCC:

guru@srap08dxoh:~/apr-1.5.1> gcc -v
Using built-in specs.
Target: i586-suse-linux
Configured with: ../configure --prefix=/usr --infodir=/usr/share/info 
--mandir=/usr/share/man --libdir=/usr/lib --libexecdir=/usr/lib 
--enable-languages=c,c++,objc,fortran,obj-c++,java,ada 
--enable-checking=release --with-gxx-include-dir=/usr/include/c++/4.3 
--enable-ssp --disable-libssp --with-bugurl=http://bugs.opensuse.org/ 
--with-pkgversion='SUSE Linux' --disable-libgcj --disable-libmudflap 
--with-slibdir=/lib --with-system-zlib --enable-__cxa_atexit 
--enable-libstdcxx-allocator=new --disable-libstdcxx-pch 
--enable-version-specific-runtime-libs --program-suffix=-4.3 
--enable-linux-futex --without-system-libunwind --with-cpu=generic 
--build=i586-suse-linux
Thread model: posix
gcc version 4.3.4 [gcc-4_3-branch revision 152973] (SUSE Linux)

After compilation the share library contains an request for GLIBC_2.10:

guru@srap08dxoh:~/apr-1.5.1> strings .libs/libapr-1.so.0.5.1 | fgrep GLIBC
GLIBC_2.1
GLIBC_2.0
GLIBC_2.3.2
GLIBC_2.2
GLIBC_2.1.3
GLIBC_2.10
GLIBC_2.9
GLIBC_2.3.3
GLIBC_2.1.2
GLIBC_2.3
GLIBC_2.1.1
GLIBC_2.4


Start with a more informative command:

nm libapr-1.so.0.5.1 | grep GLIBC

This will show you which symbol is taken with which version from GLIBC. 
Let us know the result, especially the 2.10 ones.


I don't get any 2.10 dependencies for a 32 Bit SLES 10 system (with old 
patch levels), but I do get a 2.10 dependency on SLES 11, namely for


accept4@@GLIBC_2.10

Presence of the function accept4() is checked by configure and noted via 
HAVE_ACCEPT4 in include/arch/unix/apr_private.h. If it is found it is 
used in network_io/unix/sockets.c.


You can overwrite the configure check (thus disabling the use of 
accept4()) via exporting an environment variable named apr_cv_accept4 
with value "no" before running configure.


The result of the accept4() check in configure is logged in config.log. 
you can check for the result via


% grep apr_cv_accept4 config.log
apr_cv_accept4=no
% grep HAVE_ACCEPT4 include/arch/unix/apr_private.h
/* #undef HAVE_ACCEPT4 */

Whether in your case it actually is accept4() which is the problem or 
something else can be checked using the above nm command.


Regards,

Rainer

Re: [VOTE] Release APR-util 1.5.4

[Rainer Jung]

Am 19.09.2014 um 18:37 schrieb wr...@rowe-clan.net:

Aren't most of these issues resolved by correctly using pkg-config where
it is available?


pkg-config doesn't help for

- crypto configure for OpenSSL expected to fail at least on Solaris,
   because when linking against the libssl we need
   the additional flags "-ldl -lsocket -lnsl". Currently
   there's no way to fix this apart from hacking configure.
   For Linux I'm not sure, but likely you'll need "-ldl".

- configure fails for Berkeley DB in non-standard path,
   because it doesn't add an rpath to conftest and then tries
   to run the compiled conftest binary.

- no rpath for ldap, mysql and crypto dso extensions. So no
   way to run the result without LD_LIBRARY_PATH.
   Interesting: the m4 file does add an rpath only for oracle.
   Since there's no platform independent way to add rpath,
   we might want to remove it everywhere (and users would need
   to use the respective LDADD variables).

- no rpath for ldap, mysql and crypto dependency libs when building
  without dso support, e.g. when there's only one big apr util library.

Using pkg-config does enter the right rpath to the extension libs for 
expat, sqlite, apr and oracle, so no LDADD needed there.



In any case, that isn't a showstopper against 1.5.4 IMHO, since these
don't represent regressions AFAIK.


Agreed, that's why I voted +1.

Rainer

Re: [VOTE] Release APR-util 1.5.4

[Rainer Jung]

Am 17.09.2014 um 01:47 schrieb Jeff Trawick:

Tarballs/zipfiles are at http://apr.apache.org/dev/dist/


Shortcut to CHANGES files:

http://apr.apache.org/dev/dist/CHANGES-APR-UTIL-1.5

http://apr.apache.org/dev/dist/CHANGES-APR-UTIL-1.5
.4

autoconf version: 2.69 (same as recent apr and apr-util)
libtool version: 2.4.2 (same as recent apr and apr-util)

+/-1
[+1] Release apr-util 1.5.4 as GA


No regressions, +1 for release.

Summary
===

- some libtool m4 files not deleted by buildconf (see below)

- crypto configure for OpenSSL expected to fail at least on Solaris,
   because when linking against the libssl we need
   the additional flags "-ldl -lsocket -lnsl". Currently
   there's no way to fix this apart from hacking configure.
   For Linux I'm not sure, but likely you'll need "-ldl".

- LDADD flags are typically not respected during configure,
  only during the make.

- configure fails for Berkeley DB in non-standard path,
   because it doesn't add an rpath to conftest and then tries
   to run the compiled conftest binary.

- no rpath for ldap, mysql and crypto dso extensions. So no
   way to run the result without LD_LIBRARY_PATH.
   Interesting: the m4 file does add an rpath only for oracle.
   Since there's no platform independent way to add rpath,
   we might want to remove it everywhere (and users would need
   to use the respective LDADD variables).

- no rpath for dependency libs when building without dso support,
   e.g. when the ldap, crypto, mysql, oracle, sqlite etc. extensions
   are all in the big apr util library.

- testall often fails in testcrypto when testing nss passphrase
  (see below).


Details
===

- svn compared with gz, bz2 and zip only expected differences
   except for the following in the directory xml/expat/conftools
   which are not a blocker;
   IMHO we could remove those at the and of buildconf
 libtool.m4
 lt~obsolete.m4
 ltoptions.m4
 ltsugar.m4
 ltversion.m4

- files signed, checksums correct

- built and made check on the following platforms:

   - Solaris 8 and 10 Sparc (gcc 4.1.2 resp. 4.9.0)
   - SuSE Linux Enterprise 10 32 Bit and 10 and 11 64 Bit
   - RedHat Enterprise Linux 5 and 6, 64 Bit

   - using all combinations of:

 - apr 1.5.1
 - expat builtin / 2.1.0
 - OpenSSL 1.0.1i (plus a few patches)
 - dso disable / enable
 - Berkeley DB 6.1.19
 - sqlite 3.8.6
 - mysql 6.0.2 (only Solaris)
 - oracle 11.2.0.2.0 (Solaris 10), resp. 10.2.0.5.0 (Solaris 8)
 - platform nss (Solaris 10 and RHEL 6)

- make check for the successful build ran fine for 168 builds,
  but failed for 56 builds:

  - 56 failures in testcrypto
 testing crypto for nss gives errors on Solaris 10 and on RHEL
 where I had built crypto with nss and OpenSSL (no nss build
 on the other platforms). For Solaris 10 and RHEL 6 only if
 dso is disabled, for RHEL 5 also if dso is enabled.
 For details see my 1.5.1 vote mail.

- checked httpd builds for 2.2.29 and 2.4.10 against this version
  on Solaris 10, SLES 11 and RHEL 6:
  - builds succeed
  - test suite runs without regressions

Regards,

Rainer