Re: 1.6.0 release candidates
Thanks Rainer for the detailed report (as usual).
On Mon, Apr 17, 2017 at 2:01 PM, Rainer Jung wrote:
>
> c) Failure to compile apr on Solaris 8
> --
>
> Using gcc 4.1.2 compiling locks/unix/proc_mutex.c:
>
> In function 'proc_mutex_pthread_create':
> 607: error: lvalue required as left operand of assignment
> In function 'proc_mutex_pthread_acquire_ex':
> 711: error: lvalue required as left operand of assignment
> 790: warning: implicit declaration of function 'pthread_mutex_timedlock'
> In function 'proc_mutex_pthread_release':
> 868: error: lvalue required as left operand of assignment
> In function 'proc_mutex_pthread_cond_create':
> 945: error: lvalue required as left operand of assignment
>
> Concerning pthread_mutex_timedlock: that platform has
> HAVE_PTHREAD_CONDATTR_SETPSHARED set to 1 but HAVE_PTHREAD_MUTEX_TIMEDLOCK
> is not defined (and pthread_mutex_timedlock not available). So
> APR_USE_PROC_PTHREAD_MUTEX_COND is defined. So in line 688 we enter the
> "#else" branch which ends in line 815. That code includes a call to
> pthread_mutex_timedlock.
>
> Concerning the "lvalue required as left operand of assignment" the lines are
> always of the form
>
> proc_pthread_mutex_cond_locked(SOME_MUTEX) = SOME_INT;
>
> and proc_pthread_mutex_cond_locked(m) is defined as
>
> ((m)->pthread_refcounting ? proc_pthread_cast(m)->cond_locked : -1)
>
> which indeed doesn't look like a good lvalue.
Sorry about that, I thought I had tested the
APR_USE_PROC_PTHREAD_MUTEX_COND case with some #undef's on Linux but
it seems I missed it for my latest changes.
All the above hopefuly fixes with r1791718 and r1791728, both
backported to 1.6.x.
>
> d) Hang during APR make check on Solaris 10 (testprocmutex)
> ---
>
> pthread_mutex_timedlock() hangs when the current thread already has locked
> the mutex.
>
> GDB says:
>
> #0 0xff14ca00 in ___lwp_mutex_timedlock () from /lib/libc.so.1
> No symbol table info available.
> #1 0xff13fde8 in mutex_lock_kernel () from /lib/libc.so.1
> No symbol table info available.
> #2 0xff140f4c in stall () from /lib/libc.so.1
> No symbol table info available.
> #3 0xff1416dc in mutex_lock_internal () from /lib/libc.so.1
> No symbol table info available.
> #4 0xff141cc0 in pthread_mutex_timedlock () from /lib/libc.so.1
> No symbol table info available.
> #5 0xff369310 in proc_mutex_pthread_acquire_ex (mutex=0xb4ff8,
> timeout=1492421091353146)
> at .../locks/unix/proc_mutex.c:790
> abstime = {tv_sec = 1492421091, tv_nsec = 353146000}
> rv =
> #6 0xff36a500 in apr_proc_mutex_timedlock (mutex=0xb4ff8, timeout=1) at
> .../locks/unix/proc_mutex.c:1558
> No locals.
> #7 0x00029f90 in test_exclusive (lockname=0x0, mech=0xffbffad8,
> tc=0xffbffa70) at .../test/testprocmutex.c:197
> child = {0xb5118, 0xb5128, 0xb5138, 0xb5148, 0xb5158, 0xb5168}
> rv = 0
> n = 0
> #8 proc_mutex (tc=0xffbffa70, data=0xffbffad8) at
> .../test/testprocmutex.c:243
> rv =
> shmname = 0x37ad0 "tpm.shm"
> shm = 0xb4fc8
> #9 0x00015a4c in abts_run_test (ts=ts@entry=0xb7b70, f=f@entry=0x29ccc
> , value=value@entry=0xffbffad8)
> at .../test/abts.c:171
> tc = {failed = 0, suite = 0xb76a8}
> ss = 0xb76a8
> #10 0x0002a26c in testprocmutex (suite=0xb7b70) at
> .../test/testprocmutex.c:274
> lockmechs = {{num = APR_LOCK_DEFAULT, name = 0x37d70 "default"},
> {num = APR_LOCK_SYSVSEM, name = 0x37d78 "sysvsem"}, {num =
> APR_LOCK_POSIXSEM, name = 0x37d80 "posix"},
> {num = APR_LOCK_FCNTL, name = 0x37d88 "fcntl"}, {num =
> APR_LOCK_PROC_PTHREAD, name = 0x37d90 "proc_pthread"}, {num =
> APR_LOCK_DEFAULT_TIMED,
> name = 0x37da0 "default_timed"}}
> #11 0x000336c0 in main (argc=2, argv=) at .../test/abts.c:429
> i =
> list_provided =
> suite = 0xb7b70
>
> and the timestamp given is Mon Apr 17 11:24:51 2017, which looks OK. So for
> some reason the timedlock doesn't time out when trying to acquire the
> proc_mutex a second time.
>
> truss (Solaris variant of strace) shows for the first successful call:
>
> lwp_mutex_timedlock(0xFF1D, 0xFFBFF880) = 0
> mutex type: USYNC_PROCESS|LOCK_PRIO_INHERIT|LOCK_ROBUST
> timeout: 0.0 sec
>
> and then for the hanging subsequent call by the same thread:
>
> lwp_mutex_timedlock(0xFF1D, 0xFFBFF880) Err#45 EDEADLK
> mutex type: USYNC_PROCESS|LOCK_PRIO_INHERIT|LOCK_ROBUST
> timeout: 0.0 sec
> lwp_mutex_timedlock(0xFF1B5778, 0x) (sleeping...)
> mutex type: USYNC_THREAD
>
> Assuming the Solaris code is similar to:
>
> https://github.com/OpenIndiana/illumos-gate/blame/master/usr/src/lib/libc/port/threads/synch.c
>
> maybe my Solaris 10 does not have the following code change in it:
>
> https://github.com/OpenIndiana/illumos-gate/commit/f52756fb59521fc0f684db03ee24da2a1d12a52a
>
> "67387
buildbot success in on apr-x64-macosx-trunk
The Buildbot has detected a restored build on builder apr-x64-macosx-trunk while building . Full details are available at: https://ci.apache.org/builders/apr-x64-macosx-trunk/builds/4 Buildbot URL: https://ci.apache.org/ Buildslave for this Build: svn-x64-macosx-dgvrs Build Reason: The AnyBranchScheduler scheduler named 'on-apr-commit' triggered this build Build Source Stamp: [branch apr/apr/trunk] 1791728 Blamelist: ylavic Build succeeded! Sincerely, -The Buildbot
buildbot failure in on apr-x64-macosx-trunk
The Buildbot has detected a new failure on builder apr-x64-macosx-trunk while building . Full details are available at: https://ci.apache.org/builders/apr-x64-macosx-trunk/builds/3 Buildbot URL: https://ci.apache.org/ Buildslave for this Build: svn-x64-macosx-dgvrs Build Reason: The AnyBranchScheduler scheduler named 'on-apr-commit' triggered this build Build Source Stamp: [branch apr/apr/trunk] 1791718 Blamelist: ylavic BUILD FAILED: failed Test Sincerely, -The Buildbot
Re: 1.6.0 release candidates
On Mon, Apr 17, 2017 at 11:06 AM, Nick Kew wrote: > On Mon, 2017-04-17 at 14:01 +0200, Rainer Jung wrote: > > Thanks for the comments. > >> my test results: most important is failure to compile on Solaris 8 and a >> hang during make test on Solaris 10, as well as your key seeming to be >> revoked. See below. >> >> 1) Formal observations >> == >> >> - MD5 and SHA1 missing > > Indeed. Do you think those should be encouraged, as they don't > actually give any more security than a simple checksum? IMO, MD5 (supported on any box) and SHA256 (supported only with more modern distros) still make sense. No reason to add SHA1, the MD5 is enough for non-deliberate integrity validation. SHA256 can be enough if the fingerprint is from https://apr.apache.org/dist/apr/ but isn't a recommended substitute for a valid .asc PGP validation. >> - zip archives not named "win32-src.zip" as previously See below >> - CHANGES-APR-1.6 and CHANGES-APR-UTIL-1.6 missing That could happen when you roll (I do this because I have the two tarballs unpacked at the time), but can just as easily happen when you are publishing the release. The contents are in the tarballs for review already. >> - HEADER.html and README.html missing I see in http://apr.apache.org/dev/dist/ NOTICE: This file store contains only release candidates, not released tarballs. These are only provided by the dev@apr.apache.org list's Release Managers to consider release candidates prior to moving them to their proper home http://www.apache.org/dist/apr/ so this is fine where we've traditionally posted the candidates - they don't go in the tarball. That URL corresponds to https://dist.apache.org/repos/dist/dev/apr/ >> - Announcement1.6.txt and Announcement1.6.html missing > > AFAICS those don't feature in the tarballs? I looked at > the 1.5.latest of APR/APU and I don't see them. Typically we keep a copy of these in dev/dist/ for massaging the release messages. They don't go in the tarball. Typically we simply copy (or svn mv) the release and these extra artifacts over from https://dist.apache.org/repos/dist/dev/apr/ to https://dist.apache.org/repos/dist/release/apr/ when it is time to populate the mirrors. >> - Key B87F79A9 missing from KEYS file > > Good point. Since ASF now auto-generates per-project keys files > daily, why maintain a separate KEYS file? > >> - Key B87F79A9 is listed as "revoked: 2016-08-16" in key server > > I saw that key after uploading the tarballs! > That key has a wrong creation date and wrong fingerprint: > it's an imposter who created a clash (I recollect discussing > that, though I only just found out that a fake with my name > on it was "out there")! My real key is in > http://people.apache.org/keys/group/apr.asc and > http://people.apache.org/keys/committer/ > (along with my old 1024-bit key, which I can also use if this > is creating confusion). > >> - ".svn" directory included, that's unusual >> - STATUS file included, that's unusual > > It's a fair cop, guv! > >> - Unix build files configure, *.spec, build-outputs.mk, build/*.sh, >>build/l*.m4 (libtool m4 files) and the copies of the apr build/*.m4 >>in apu are included in zip (that's new, but IMHO OK) > > Ah, right. Yep, that's probably not such a good idea. Right - this is why they were named -win32-src.zip, to remind folks that they can't find the unix tools, even if they were to switch the lineends back to lf. > Do we in fact still need those in the Unix tarballs? Are there > platforms out there with the toolchain for configure/make but not > for buildconf? YES! The whole autocruft is very hard to get right on older solaris, hpux, aix and other operating systems that just aren't Linux. Can't comment on BSD compatibility of autocruft. >> - no more bundled expat: we need to note that in the release notes > > Agreed, that was certainly on the agenda for the announcement. +1 to include in Announcement (and probably on the site download reference for the package). A pointer to the project would be good; http://expat.sourceforge.net/ (soon to be github, but there isn't much there yet.)
Re: 1.6.0 release candidates
Hi Nick, Am 17.04.2017 um 18:06 schrieb Nick Kew: On Mon, 2017-04-17 at 14:01 +0200, Rainer Jung wrote: Thanks for the comments. my test results: most important is failure to compile on Solaris 8 and a hang during make test on Solaris 10, as well as your key seeming to be revoked. See below. 1) Formal observations == - MD5 and SHA1 missing Indeed. Do you think those should be encouraged, as they don't actually give any more security than a simple checksum? I thought those are not for security but to easily detect transfer errors. Only the pgp signature is for security. The hashes still seem to be frequently provided. - zip archives not named "win32-src.zip" as previously - CHANGES-APR-1.6 and CHANGES-APR-UTIL-1.6 missing - HEADER.html and README.html missing - Announcement1.6.txt and Announcement1.6.html missing AFAICS those don't feature in the tarballs? I looked at the 1.5.latest of APR/APU and I don't see them. Right, those files will not be part of the artefacts but exist in the download directory, e.g.: https://dist.apache.org/repos/dist/release/apr/ or http://archive.apache.org/dist/apr/ - Key B87F79A9 missing from KEYS file Good point. Since ASF now auto-generates per-project keys files daily, why maintain a separate KEYS file? Didn't know about that feature. Can it also be used to check historic signatures, e.g. of old releases? At least we would have to rephrase http://apr.apache.org/download.cgi the KEYS file is mentioned there. - Key B87F79A9 is listed as "revoked: 2016-08-16" in key server I saw that key after uploading the tarballs! That key has a wrong creation date and wrong fingerprint: it's an imposter who created a clash (I recollect discussing that, though I only just found out that a fake with my name on it was "out there")! My real key is in http://people.apache.org/keys/group/apr.asc and http://people.apache.org/keys/committer/ (along with my old 1024-bit key, which I can also use if this is creating confusion). Thanks for correcting this. - ".svn" directory included, that's unusual - STATUS file included, that's unusual It's a fair cop, guv! - Unix build files configure, *.spec, build-outputs.mk, build/*.sh, build/l*.m4 (libtool m4 files) and the copies of the apr build/*.m4 in apu are included in zip (that's new, but IMHO OK) Ah, right. Yep, that's probably not such a good idea. Do we in fact still need those in the Unix tarballs? Are there platforms out there with the toolchain for configure/make but not for buildconf? I always thought about deleting the build/l*.m4 files coming from libtool while running buildconf at the end of buildconf. I think they are not needed for building and if you rerun buildoncf they will again get copied from the libtool installation. IMHO we should stick to configure/make as the normal build procedure and not demand people to run buildconf. Users who build should not have a need to care about libtool and auto tools installations. So I'd keep the files generated by buildconf for the Unix tarballs (except for the build/l*.m4 files as mentioned above). - no more bundled expat: we need to note that in the release notes Agreed, that was certainly on the agenda for the announcement. 2) Compile and run == a) New apr configure options b) Changes in apr-util configure options Useful for release notes :) c) Failure to compile apr on Solaris 8 -- Damn. That looks like a showstopper for release. But shouldn't be too hard, since you've tracked down the cause. Yes I think that one should get fixed. which indeed doesn't look like a good lvalue. Indeedie. d) Hang during APR make check on Solaris 10 (testprocmutex) --- pthread_mutex_timedlock() hangs when the current thread already has locked the mutex. Hmmm, OK. We knew the timedlock stuff had potential to blow up. I guess that points back to the plan of making it a config option. I'll try to dig some more info whether the problem can be fixed by an OS patch for Solaris 10. At least it shows the potential for problems. I'm a bit undecided on the config option, because that means we'll have builds with subtle API differences. Most of the other APR config options are not a good comparison for a consistent decision. I'm out this evening, but will hopefully find time tomorrow to revisit these problems. And I need to do some more digging around that bogus PGP key! Thanks! Rainer
Re: 1.6.0 release candidates
On Mon, 2017-04-17 at 14:01 +0200, Rainer Jung wrote: Thanks for the comments. > my test results: most important is failure to compile on Solaris 8 and a > hang during make test on Solaris 10, as well as your key seeming to be > revoked. See below. > > 1) Formal observations > == > > - MD5 and SHA1 missing Indeed. Do you think those should be encouraged, as they don't actually give any more security than a simple checksum? > - zip archives not named "win32-src.zip" as previously > - CHANGES-APR-1.6 and CHANGES-APR-UTIL-1.6 missing > - HEADER.html and README.html missing > - Announcement1.6.txt and Announcement1.6.html missing AFAICS those don't feature in the tarballs? I looked at the 1.5.latest of APR/APU and I don't see them. > - Key B87F79A9 missing from KEYS file Good point. Since ASF now auto-generates per-project keys files daily, why maintain a separate KEYS file? > - Key B87F79A9 is listed as "revoked: 2016-08-16" in key server I saw that key after uploading the tarballs! That key has a wrong creation date and wrong fingerprint: it's an imposter who created a clash (I recollect discussing that, though I only just found out that a fake with my name on it was "out there")! My real key is in http://people.apache.org/keys/group/apr.asc and http://people.apache.org/keys/committer/ (along with my old 1024-bit key, which I can also use if this is creating confusion). > - ".svn" directory included, that's unusual > - STATUS file included, that's unusual It's a fair cop, guv! > - Unix build files configure, *.spec, build-outputs.mk, build/*.sh, >build/l*.m4 (libtool m4 files) and the copies of the apr build/*.m4 >in apu are included in zip (that's new, but IMHO OK) Ah, right. Yep, that's probably not such a good idea. Do we in fact still need those in the Unix tarballs? Are there platforms out there with the toolchain for configure/make but not for buildconf? > - no more bundled expat: we need to note that in the release notes Agreed, that was certainly on the agenda for the announcement. > 2) Compile and run > == > > a) New apr configure options > b) Changes in apr-util configure options Useful for release notes :) > c) Failure to compile apr on Solaris 8 > -- Damn. That looks like a showstopper for release. But shouldn't be too hard, since you've tracked down the cause. > which indeed doesn't look like a good lvalue. Indeedie. > d) Hang during APR make check on Solaris 10 (testprocmutex) > --- > > pthread_mutex_timedlock() hangs when the > current thread already has > locked the mutex. Hmmm, OK. We knew the timedlock stuff had potential to blow up. I guess that points back to the plan of making it a config option. I'm out this evening, but will hopefully find time tomorrow to revisit these problems. And I need to do some more digging around that bogus PGP key! -- Nick Kew
Re: 1.6.0 release candidates
On 17.04.2017 14:01, Rainer Jung wrote: > - ".svn" directory included, that's unusual Always 'svn export' the tag, not 'svn checkout' ...
Re: 1.6.0 release candidates
Hi Nick,
Am 16.04.2017 um 01:12 schrieb Nick Kew:
Today I have tagged both APR and APR-UTIL 1.6.0
and rolled tarballs of release candidates in
the full choice of formats provided for 1.5.latest
on our pages.
Please download and test, from
http://people.apache.org/~niq/apr/
my test results: most important is failure to compile on Solaris 8 and a
hang during make test on Solaris 10, as well as your key seeming to be
revoked. See below.
1) Formal observations
==
- MD5 and SHA1 missing
- zip archives not named "win32-src.zip" as previously
- CHANGES-APR-1.6 and CHANGES-APR-UTIL-1.6 missing
- HEADER.html and README.html missing
- Announcement1.6.txt and Announcement1.6.html missing
- Key B87F79A9 missing from KEYS file
- Key B87F79A9 is listed as "revoked: 2016-08-16" in key server
- ".svn" directory included, that's unusual
- STATUS file included, that's unusual
- Unix build files configure, *.spec, build-outputs.mk, build/*.sh,
build/l*.m4 (libtool m4 files) and the copies of the apr build/*.m4
in apu are included in zip (that's new, but IMHO OK)
- no more bundled expat: we need to note that in the release notes
2) Compile and run
==
a) New apr configure options
--enable-allocator-guard-pages Use guard pages in apr_allocator
(implies --enable-allocator-uses-mmap)
--enable-pool-concurrency-check Check for concurrent usage of memory
pools
--with-valgrind[=DIR] Enable code to teach valgrind about apr pools
(optionally: set path to valgrind headers)
b) Changes in apr-util configure options
- new --with-commoncrypto=DIR specify location of CommonCrypto
- removed --with-freetds=DIR specify FreeTDS location
- changed --with-expat=DIR from "specify Expat location, or
'builtin'" to "specify Expat location"
c) Failure to compile apr on Solaris 8
--
Using gcc 4.1.2 compiling locks/unix/proc_mutex.c:
In function 'proc_mutex_pthread_create':
607: error: lvalue required as left operand of assignment
In function 'proc_mutex_pthread_acquire_ex':
711: error: lvalue required as left operand of assignment
790: warning: implicit declaration of function 'pthread_mutex_timedlock'
In function 'proc_mutex_pthread_release':
868: error: lvalue required as left operand of assignment
In function 'proc_mutex_pthread_cond_create':
945: error: lvalue required as left operand of assignment
Concerning pthread_mutex_timedlock: that platform has
HAVE_PTHREAD_CONDATTR_SETPSHARED set to 1 but
HAVE_PTHREAD_MUTEX_TIMEDLOCK is not defined (and pthread_mutex_timedlock
not available). So APR_USE_PROC_PTHREAD_MUTEX_COND is defined. So in
line 688 we enter the "#else" branch which ends in line 815. That code
includes a call to pthread_mutex_timedlock.
Concerning the "lvalue required as left operand of assignment" the lines
are always of the form
proc_pthread_mutex_cond_locked(SOME_MUTEX) = SOME_INT;
and proc_pthread_mutex_cond_locked(m) is defined as
((m)->pthread_refcounting ? proc_pthread_cast(m)->cond_locked : -1)
which indeed doesn't look like a good lvalue.
d) Hang during APR make check on Solaris 10 (testprocmutex)
---
pthread_mutex_timedlock() hangs when the current thread already has
locked the mutex.
GDB says:
#0 0xff14ca00 in ___lwp_mutex_timedlock () from /lib/libc.so.1
No symbol table info available.
#1 0xff13fde8 in mutex_lock_kernel () from /lib/libc.so.1
No symbol table info available.
#2 0xff140f4c in stall () from /lib/libc.so.1
No symbol table info available.
#3 0xff1416dc in mutex_lock_internal () from /lib/libc.so.1
No symbol table info available.
#4 0xff141cc0 in pthread_mutex_timedlock () from /lib/libc.so.1
No symbol table info available.
#5 0xff369310 in proc_mutex_pthread_acquire_ex (mutex=0xb4ff8,
timeout=1492421091353146)
at .../locks/unix/proc_mutex.c:790
abstime = {tv_sec = 1492421091, tv_nsec = 353146000}
rv =
#6 0xff36a500 in apr_proc_mutex_timedlock (mutex=0xb4ff8, timeout=1) at
.../locks/unix/proc_mutex.c:1558
No locals.
#7 0x00029f90 in test_exclusive (lockname=0x0, mech=0xffbffad8,
tc=0xffbffa70) at .../test/testprocmutex.c:197
child = {0xb5118, 0xb5128, 0xb5138, 0xb5148, 0xb5158, 0xb5168}
rv = 0
n = 0
#8 proc_mutex (tc=0xffbffa70, data=0xffbffad8) at
.../test/testprocmutex.c:243
rv =
shmname = 0x37ad0 "tpm.shm"
shm = 0xb4fc8
#9 0x00015a4c in abts_run_test (ts=ts@entry=0xb7b70, f=f@entry=0x29ccc
, value=value@entry=0xffbffad8)
at .../test/abts.c:171
tc = {failed = 0, suite = 0xb76a8}
ss = 0xb76a8
#10 0x0002a26c in testprocmutex (suite=0xb7b70) at
.../test/testprocmutex.c:274
lockmechs = {{num = APR_LOCK_DEFAULT, name = 0x37d70
"default"}, {num = APR_LOCK_SYSVSEM, name = 0x37
