It seems solved now, reporting back. It looks to me like in February, when the certificate renewal failed, I had hit the bug described here: https://www.redhat.com/archives/freeipa-users/2016-February/msg00441.html
Yesterday I updated the packages, including the fix to this bug, but then I still had an expired certificate. Which didn't allow to complete ipa-server-upgrade. Went back in time, asked certmonger to renew, but I was then missing certificate profiles, because the upgrade was not completed. Now however the certificate was valid, because the date was changed. With that, I could manually run ipa-server-upgrade, which successfully imported all profiles. Restarted ipa, restarted certmonger, got new certificates. Went back to today's date, restarted ipa, and everything seems fine. On Wed, 7 Jun 2017 at 23:25 Roberto Cornacchia <roberto.cornacc...@gmail.com> wrote: > A relatively good news: > The current error (Insufficient access: Principal 'HTTP/ > spinque04.hq.spinque....@hq.spinque.com' is not permitted to use CA '.' > with profile 'caIPAserviceCert' for certificate issuance.) might not be > due to the package upgrade. > > I looked at the journal of 16 Feb 2017 (28 days before the expiration > date): certmonger correctly tries to renew the certificate but fails with > the exact same error as I have now. So this explains why I ended up with an > expired certificate in the first place. > > So hopefully I'm back to the original issue that caused all this. Any help > is highly appreciated. > > > On Wed, 7 Jun 2017 at 19:15 Rob Crittenden <rcrit...@redhat.com> wrote: > >> Roberto Cornacchia via FreeIPA-users wrote: >> > Sorry for accidentally dropping freeipa-users. >> > >> > I was impatient so went back in time before your answer, but I did chose >> > a good date >> > >> > Before this, I had the following two entries with an expired date: >> > >> > Request ID '20150316184508': >> > status: NEED_TO_SUBMIT >> > ca-error: Error setting up ccache for "host" service on client using >> > default keytab: Cannot contact any KDC for requested realm. >> > >> > Request ID '20150316184529': >> > status: CA_UNREACHABLE >> > ca-error: Error setting up ccache for "host" service on client using >> > default keytab: Cannot contact any KDC for requested realm. >> > >> > After restarting certmonger, I have: >> > >> > Request ID '20150316184508': >> > status: MONITORING >> > ca-error: Server at https://spinque04.hq.spinque.com/ipa/xml >> > denied our request, giving up: 2100 (RPC failed at server. Insufficient >> > access: Principal 'ldap/spinque04.hq.spinque....@hq.spinque.com >> > <mailto:spinque04.hq.spinque....@hq.spinque.com>' is not permitted to >> > use CA '.' with profile 'caIPAserviceCert' for certificate issuance.). >> > >> > Request ID '20150316184529': >> > status: MONITORING >> > ca-error: Server at https://spinque04.hq.spinque.com/ipa/xml >> > denied our request, giving up: 2100 (RPC failed at server. Insufficient >> > access: Principal 'HTTP/spinque04.hq.spinque....@hq.spinque.com >> > <mailto:spinque04.hq.spinque....@hq.spinque.com>' is not permitted to >> > use CA '.' with profile 'caIPAserviceCert' for certificate issuance.). >> >> I think this is a side-effect of updating the packages with expired >> certs (you are about half upgraded right now). The new CA ACL rules >> don't seem to have been applied. I'm not sure what the safest course in, >> maybe Flo or Fraser know. >> >> rob >> >> > >> > The journal shows continuous Tomcat errors such as this: >> > >> > Mar 01 00:09:28 spinque04.hq.spinque.com >> > <http://spinque04.hq.spinque.com> named-pkcs11[5692]: validating >> > docs.oracle.com/CNAME <http://docs.oracle.com/CNAME>: bad cache hit >> > (oracle.com/DS <http://oracle.com/DS>) >> > Mar 01 00:09:28 spinque04.hq.spinque.com >> > <http://spinque04.hq.spinque.com> named-pkcs11[5692]: broken trust >> chain >> > resolving 'docs.oracle.com/A/IN <http://docs.oracle.com/A/IN>': >> 8.8.8.8#53 >> > Mar 01 00:09:28 spinque04.hq.spinque.com >> > <http://spinque04.hq.spinque.com> named-pkcs11[5692]: validating >> > docs.oracle.com/CNAME <http://docs.oracle.com/CNAME>: bad cache hit >> > (oracle.com/DS <http://oracle.com/DS>) >> > Mar 01 00:09:28 spinque04.hq.spinque.com >> > <http://spinque04.hq.spinque.com> named-pkcs11[5692]: broken trust >> chain >> > resolving 'docs.oracle.com/AAAA/IN <http://docs.oracle.com/AAAA/IN>': >> > 8.8.8.8#53 >> > Mar 01 00:09:32 spinque04.hq.spinque.com >> > <http://spinque04.hq.spinque.com> server[5912]: Mar 01, 2017 12:09:32 >> AM >> > org.apache.catalina.core.ContainerBase backgroundProcess >> > Mar 01 00:09:32 spinque04.hq.spinque.com >> > <http://spinque04.hq.spinque.com> server[5912]: WARNING: Exception >> > processing realm com.netscape.cms.tomcat.ProxyRealm@4077b502 background >> > process >> > Mar 01 00:09:32 spinque04.hq.spinque.com >> > <http://spinque04.hq.spinque.com> server[5912]: >> > java.lang.NullPointerException >> > Mar 01 00:09:32 spinque04.hq.spinque.com >> > <http://spinque04.hq.spinque.com> server[5912]: at >> > >> com.netscape.cms.tomcat.ProxyRealm.backgroundProcess(ProxyRealm.java:109) >> > Mar 01 00:09:32 spinque04.hq.spinque.com >> > <http://spinque04.hq.spinque.com> server[5912]: at >> > >> org.apache.catalina.core.ContainerBase.backgroundProcess(ContainerBase.java:1154) >> > Mar 01 00:09:32 spinque04.hq.spinque.com >> > <http://spinque04.hq.spinque.com> server[5912]: at >> > >> org.apache.catalina.core.StandardContext.backgroundProcess(StandardContext.java:5697) >> > Mar 01 00:09:32 spinque04.hq.spinque.com >> > <http://spinque04.hq.spinque.com> server[5912]: at >> > >> org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1377) >> > Mar 01 00:09:32 spinque04.hq.spinque.com >> > <http://spinque04.hq.spinque.com> server[5912]: at >> > >> org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1381) >> > Mar 01 00:09:32 spinque04.hq.spinque.com >> > <http://spinque04.hq.spinque.com> server[5912]: at >> > >> org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1381) >> > Mar 01 00:09:32 spinque04.hq.spinque.com >> > <http://spinque04.hq.spinque.com> server[5912]: at >> > >> org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1349) >> > Mar 01 00:09:32 spinque04.hq.spinque.com >> > <http://spinque04.hq.spinque.com> server[5912]: at >> > java.lang.Thread.run(Thread.java:745) >> > >> > On Wed, 7 Jun 2017 at 16:22 Rob Crittenden <rcrit...@redhat.com >> > <mailto:rcrit...@redhat.com>> wrote: >> > >> > Adding freeipa-users back. >> > >> > Roberto Cornacchia wrote: >> > > Just to confirm, this is indeed the expired certificate >> > > >> > > $ getcert list -d /etc/httpd/alias -n Server-Cert >> > > Number of certificates and requests being tracked: 8. >> > > Request ID '20150316184529': >> > > status: CA_UNREACHABLE >> > > ca-error: Error setting up ccache for "host" service on client >> using >> > > default keytab: Cannot contact any KDC for requested realm. >> > > stuck: no >> > > key pair storage: >> > > >> > >> type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS >> > > Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt' >> > > certificate: >> > > >> > >> type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS >> > > Certificate DB' >> > > CA: IPA >> > > issuer: CN=Certificate Authority,O=HQ.SPINQUE.COM >> > <http://HQ.SPINQUE.COM> <http://HQ.SPINQUE.COM> >> > > subject: CN=spinque04.hq.spinque.com < >> http://spinque04.hq.spinque.com> >> > > <http://spinque04.hq.spinque.com>,O=HQ.SPINQUE.COM >> > <http://HQ.SPINQUE.COM> <http://HQ.SPINQUE.COM> >> > > expires: 2017-03-16 18:45:29 UTC >> > > key usage: >> > digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment >> > > eku: id-kp-serverAuth,id-kp-clientAuth >> > > pre-save command: >> > > post-save command: /usr/lib64/ipa/certmonger/restart_httpd >> > > track: yes >> > > auto-renew: yes >> > >> > Since you have Apache up and limping along I'd first try: >> > >> > # getcert resubmit -i 20150316184529 >> > >> > It is very possible that the tool will reject the server cert since >> it >> > is expired. If that happens then you'll need to set time back. >> > >> > I'd encourage you to run: getcert list | grep expires >> > >> > This will show you what certs need to be renewed and should give >> some >> > insight into the window of opportunity for setting the date back. >> > >> > Assuming it's just the Apache cert that is expired I'd try setting >> the >> > date back to March 15, restart IPA, then I'd restart certmonger or >> try >> > the about resubmit command again. >> > >> > Things can get tricky when some certs have renewed and some haven't. >> > >> > rob >> > >> > > >> > > >> > > On Wed, 7 Jun 2017 at 15:36 Roberto Cornacchia >> > > <roberto.cornacc...@gmail.com >> > <mailto:roberto.cornacc...@gmail.com> >> > <mailto:roberto.cornacc...@gmail.com >> > <mailto:roberto.cornacc...@gmail.com>>> wrote: >> > > >> > > Thanks Rob, >> > > >> > > I've seen in similar posts that you recommend to go back in >> > time and >> > > restart certmonger. Would it work in this case? >> > > >> > > >> > > On Wed, 7 Jun 2017 at 15:30 Rob Crittenden >> > <rcrit...@redhat.com <mailto:rcrit...@redhat.com> >> > > <mailto:rcrit...@redhat.com <mailto:rcrit...@redhat.com>>> >> wrote: >> > > >> > > Roberto Cornacchia via FreeIPA-users wrote: >> > > > OK, I did so and httpd restarts. >> > > > >> > > > $ openssl s_client -connect 127.0.0.1:443 >> > <http://127.0.0.1:443> >> > > <http://127.0.0.1:443> <http://127.0.0.1:443> -showcerts >> > > > CONNECTED(00000003) >> > > > depth=1 O = HQ.SPINQUE.COM <http://HQ.SPINQUE.COM> >> > <http://HQ.SPINQUE.COM> >> > > <http://HQ.SPINQUE.COM>, CN = Certificate >> > > > Authority >> > > > verify return:1 >> > > > depth=0 O = HQ.SPINQUE.COM <http://HQ.SPINQUE.COM> >> > <http://HQ.SPINQUE.COM> >> > > <http://HQ.SPINQUE.COM>, CN = >> > > > spinque04.hq.spinque.com >> > <http://spinque04.hq.spinque.com> <http://spinque04.hq.spinque.com> >> > > <http://spinque04.hq.spinque.com> >> > > > verify error:num=10:certificate has expired >> > > > notAfter=Mar 16 18:45:29 2017 GMT >> > > > verify return:1 >> > > > depth=0 O = HQ.SPINQUE.COM <http://HQ.SPINQUE.COM> >> > <http://HQ.SPINQUE.COM> >> > > <http://HQ.SPINQUE.COM>, CN = >> > > > spinque04.hq.spinque.com >> > <http://spinque04.hq.spinque.com> <http://spinque04.hq.spinque.com> >> > > <http://spinque04.hq.spinque.com> >> > > > notAfter=Mar 16 18:45:29 2017 GMT >> > > > verify return:1 >> > > > --- >> > > > Certificate chain >> > > > 0 s:/O=HQ.SPINQUE.COM/CN=spinque04.hq.spinque.com >> > <http://HQ.SPINQUE.COM/CN=spinque04.hq.spinque.com> >> > > <http://HQ.SPINQUE.COM/CN=spinque04.hq.spinque.com> >> > > > <http://HQ.SPINQUE.COM/CN=spinque04.hq.spinque.com> >> > > > i:/O=HQ.SPINQUE.COM/CN=Certificate >> > <http://HQ.SPINQUE.COM/CN=Certificate> >> > > <http://HQ.SPINQUE.COM/CN=Certificate> >> > > > <http://HQ.SPINQUE.COM/CN=Certificate> Authority >> > > > ... >> > > > >> > > > Fair enough, but why does this say it expires in 2019? >> Are >> > > they two >> > > > different certificates? >> > > > >> > > > $ getcert list -d /etc/httpd/alias -n ipaCert >> > > > Number of certificates and requests being tracked: 8. >> > > > Request ID '20160501114633': >> > > > status: MONITORING >> > > > stuck: no >> > > > key pair storage: >> > > > >> > > >> > >> type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS >> > > > Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt' >> > > > certificate: >> > > > >> > > >> > >> type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS >> > > > Certificate DB' >> > > > CA: dogtag-ipa-ca-renew-agent >> > > > issuer: CN=Certificate Authority,O=HQ.SPINQUE.COM >> > <http://HQ.SPINQUE.COM> >> > > <http://HQ.SPINQUE.COM> <http://HQ.SPINQUE.COM> >> > > > subject: CN=IPA RA,O=HQ.SPINQUE.COM >> > <http://HQ.SPINQUE.COM> <http://HQ.SPINQUE.COM> >> > > <http://HQ.SPINQUE.COM> >> > > > expires: 2019-01-26 19:41:51 UTC >> > > > key usage: >> > > >> > digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment >> > > > eku: id-kp-serverAuth,id-kp-clientAuth >> > > > pre-save command: >> > /usr/lib64/ipa/certmonger/renew_ra_cert_pre >> > > > post-save command: >> /usr/lib64/ipa/certmonger/renew_ra_cert >> > > > track: yes >> > > > auto-renew: yes >> > > > >> > > > What's the right way to solve this? >> > > >> > > You're looking at the wrong cert. >> > > >> > > # getcert list -d /etc/httpd/alias -n Server-Cert >> > > >> > > And really, you should examine all certificate status, not >> > just >> > > a single >> > > one. >> > > >> > > I was also strongly urge you to wait until all problems >> > are resolved >> > > before attempting to update packages in the future (unless >> > a package >> > > claims to fix a specific problem), particularly when it >> > comes to >> > > certificates. >> > > >> > > rob >> > > >> > >> > >> > >> > _______________________________________________ >> > FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org >> > To unsubscribe send an email to >> freeipa-users-le...@lists.fedorahosted.org >> > >> >>
_______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-le...@lists.fedorahosted.org
