Did some more investigation. I downloaded the packages that are being used on centos stream 8. First I tried my test code with their version of libssl3.so preloaded: it failed in the same way as all the others failed--not surprisingly since its version is much later than the 3.39 version where this changed.
Then, I downloaded and took a look at "dogtag-submit" from the CentOS Stream 8 (RedHat) certmonger package. As far as I can tell, their version of "dogtag-submit" is *not* linked against libcurl-nss.so at all like the version on debian sid. Instead, all their certmonger helper programs are linked against the OpenSSL version (libcurl.so.4). So, I think that we should just link these against the openssl version, as the RedHat packages do and get things to work again. This raises two other issues: - is there truly a bug in the ssl portion of the nss library? If so, this should probably be brought to the attention. - perhaps the libcurl package ought to be reconfigured such that one can install the dev packages simultaneously. Right now, libcurl-nss also makes a symlink "libcurl.so" that makes it conflict with the libcurl-openssl package to which the "libcurl.so.x.x" lib belongs to. I think that the libcurl-gnutls package might do the same thing. My next step will be do rebuild freeipa and certmonger with the libcurl-openssl-dev package in place instead of the libcurl-nss-dev and then try reinstalling. On Sat, Oct 9, 2021 at 9:59 PM Spencer Olson <olso...@umich.edu> wrote: > > Cloned nss repo and did a git bisect: the first commit that causes > problems is at the upstream merge of 3.39 (upstream/3.39). > > From a very brief perusal of the upstream changes, I see there are > some edits with respect to TLS1.3--perhaps this is the reason for our > problems--I haven't yet looked hard at all the upstream changes (or > tried to bisect the upstream repo yet). > > On Sat, Oct 9, 2021 at 12:05 PM Spencer Olson <olso...@umich.edu> wrote: > > > > Since it doesn't look like any progress has been made on this, I've > > started to work through some debugging. > > > > Right now, it looks like the problem is probably actually due to a > > change in libnss3. In fact, the problem appears to be specifically in > > libssl3.so from the libnss3 package. > > > > The problem: > > * certmonger has a hard time finishing the certificate requests > > because it can't seem to authenticate to the dogtag PKI server. > > > > Observations: > > * When certmonger attempts to request a signed certificate for the > > renewal agent, it temporarily explicitly uses the ipa-ca-agent > > certificate which has been temporarily extracted from the > > /root/ca-agent.p12 storage. > > * dogtag-submit attempts to use the CURL library to submit the > > request, subsequently approve the request, and then poll for its > > finish. > > * The initial request does not use/require an encrypted channel, but > > the approval and subsequent queries do. > > * These attempts to authenticate over this encrypted channel using > > the client certificate are rejected. > > > > Hacks & tests: > > * By creating a very small c-program that does the same CURL commands > > as dogtag-submit from the certmonger package, this same authorization > > denied can be seen. > > * By simply replacing the libssl3.so library, using either LD_PRELOAD > > or LD_LIBRARY_PATH, from a prior version, the requests succeed. As of > > now, I've tried only one other version of libssl3.so (libnss3 3.35 > > from ubuntu 18.04). > > * Also, instead of linking against libcurl-nss and manualy replacing > > the libssl3.so library, success can be found by linking to > > libcurl-gnutls or libcurl-openssl > > > > I suspect that a compile option in libnss3 has to be changed in order > > for this to work again. > > > > Still todo: > > * I haven't fully discovered which part/option from libnss3 might have changed. > > * I haven't yet successfully had libnss3 emit much > > debugging--probably have to recompile with DEBUG=1.