On 04/04/2018 12:37 PM, lejeczek via FreeIPA-users wrote:


On 04/04/18 09:36, Florence Blanc-Renaud wrote:
On 04/03/2018 08:37 PM, lejeczek wrote:


On 29/03/18 12:43, Florence Blanc-Renaud wrote:
On 03/28/2018 12:42 PM, lejeczek via FreeIPA-users wrote:
hi guys,

I fail to troubleshoot this here:

$ ipactl start --ignore-service-failures
Starting Directory Service
Starting krb5kdc Service
Starting kadmin Service
Starting named Service
Starting httpd Service
Starting ipa-custodia Service
Starting ntpd Service
Starting pki-tomcatd Service
Failed to start pki-tomcatd Service
Forced start, ignoring pki-tomcatd Service, continuing normal operation

Hi,

pki-tomcatd may fail to start when the subsystemCert cert-pki-ca did not properly get renewed. Please find more information in this blog: https://floblanc.wordpress.com/2017/09/11/troubleshooting-freeipa-pki-tomcatd-fails-to-start/

Flo

Starting ipa-otpd Service
Starting ipa-dnskeysyncd Service
ipa: INFO: The ipactl command was successful

logs in /var/log/pki/pki-tomcat:
localhost.2018-03-28.log
...
Mar 28, 2018 11:35:14 AM org.apache.catalina.core.StandardHostValve invoke
SEVERE: Exception Processing /ca/admin/ca/getStatus
javax.ws.rs.ServiceUnavailableException: Subsystem unavailable
?????????????? at com.netscape.cms.tomcat.ProxyRealm.findSecurityConstraints(ProxyRealm.java:145) ?????????????? at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:500) ?????????????? at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169) ?????????????? at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) ?????????????? at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:962) ?????????????? at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) ?????????????? at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:445) ?????????????? at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1087) ?????????????? at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:637) ?????????????? at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316) ?????????????? at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ?????????????? at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ?????????????? at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
?????????????? at java.lang.Thread.run(Thread.java:748)


in catalina.2018-03-28.log:
...
Mar 28, 2018 11:41:35 AM org.apache.catalina.core.ContainerBase backgroundProcess WARNING: Exception processing realm com.netscape.cms.tomcat.ProxyRealm@1e572093 background process
javax.ws.rs.ServiceUnavailableException: Subsystem unavailable
?????? at com.netscape.cms.tomcat.ProxyRealm.backgroundProcess(ProxyRealm.java:137) ?????? at org.apache.catalina.core.ContainerBase.backgroundProcess(ContainerBase.java:1356) ?????? at org.apache.catalina.core.StandardContext.backgroundProcess(StandardContext.java:5958) ?????? at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1542) ?????? at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1552) ?????? at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1552) ?????? at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1520)
?????? at java.lang.Thread.run(Thread.java:748)


Would you able to conclude anything from those errors? What might be a problem?

many thanks, L.
_______________________________________________
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org
To unsubscribe send an email to freeipa-users-le...@lists.fedorahosted.org


I have followed those instructions from the link and it seems that both certutil & ldap have the same certificate.
However I also see:

$ sudo journalctl -lf -o cat -u dirsrv@
...
GSSAPI client step 1
GSSAPI client step 1
GSSAPI client step 2
[03/Apr/2018:19:30:53.962565693 +0100] - ERR - slapi_ldap_bind - Error: could not send startTLS request: error -11 (Connect error) [03/Apr/2018:19:30:53.965606137 +0100] - ERR - slapi_ldap_bind - Error: could not send startTLS request: error -11 (Connect error)
GSSAPI client step 1
GSSAPI client step 1
GSSAPI client step 1
..

and in /var/log/pki/pki-tomcat/ca/debug

[03/Apr/2018:19:09:45][localhost-startStop-1]: SSLClientCertificatSelectionCB: Entering! [03/Apr/2018:19:09:45][localhost-startStop-1]: Candidate cert: Server-Cert cert-pki-ca [03/Apr/2018:19:09:45][localhost-startStop-1]: Candidate cert: caSigningCert cert-pki-ca [03/Apr/2018:19:09:45][localhost-startStop-1]: SSLClientCertificateSelectionCB: returning: null

Hi,
it looks like the subsystemCert is not picked to authenticate to the LDAP server. Can you check if the content of /etc/pki/pki-tomcat/ca/CS.cfg is also consistent: this file contains an entry for ca.subsystem.cert=MII.. that should match the cert 'subsystemCert cert-pki-ca' stored in /etc/pki/pki-tomcat/alias/ and in LDAP.

Flo

[03/Apr/2018:19:09:45][localhost-startStop-1]: SSL handshake happened
Could not connect to LDAP server host rider.private port 636 Error netscape.ldap.LDAPException: Authentication failed (48)          at com.netscape.cmscore.ldapconn.LdapBoundConnFactory.makeConnection(LdapBoundConnFactory.java:205)          at com.netscape.cmscore.ldapconn.LdapBoundConnFactory.init(LdapBoundConnFactory.java:166)          at com.netscape.cmscore.ldapconn.LdapBoundConnFactory.init(LdapBoundConnFactory.java:130)          at com.netscape.cmscore.dbs.DBSubsystem.init(DBSubsystem.java:654)          at com.netscape.cmscore.apps.CMSEngine.initSubsystem(CMSEngine.java:1176)          at com.netscape.cmscore.apps.CMSEngine.initSubsystems(CMSEngine.java:1082)
         at com.netscape.cmscore.apps.CMSEngine.init(CMSEngine.java:572)
         at com.netscape.certsrv.apps.CMS.init(CMS.java:189)
         at com.netscape.certsrv.apps.CMS.start(CMS.java:1631)
         at com.netscape.cms.servlet.base.CMSStartServlet.init(CMSStartServlet.java:117)
         at javax.servlet.GenericServlet.init(GenericServlet.java:158)
         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
         at java.lang.reflect.Method.invoke(Method.java:498)
         at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:288)          at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:285)
         at java.security.AccessController.doPrivileged(Native Method)
         at javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
         at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:320)          at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:175)          at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:124)          at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1257)          at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1182)          at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1072)          at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5368)          at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5660)          at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145)          at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:899)          at org.apache.catalina.core.ContainerBase.access$000(ContainerBase.java:133)          at org.apache.catalina.core.ContainerBase$PrivilegedAddChild.run(ContainerBase.java:156)          at org.apache.catalina.core.ContainerBase$PrivilegedAddChild.run(ContainerBase.java:145)
         at java.security.AccessController.doPrivileged(Native Method)
         at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:873)          at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:652)          at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:679)          at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1966)          at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
..




could this have some causation in expired certs?

Hi,

CA_WORKING means that certmonger's helper is trying to download the certificate from LDAP, but does not find new certs.

In topologies with multiple servers, only one server is the renewal master. When one of auditSigningCert cert-pki-ca, ocspSigningCert cert-pki-ca, subsystemCert cert-pki-ca or caSigningCert cert-pki-ca expires, the renewal master is the one that actually handles the renewal, and the other masters simply download the new certs from LDAP.

You need to check which server is your renewal master (ipa config-show | grep 'IPA CA renewal master'), then make sure that the certs were properly renewed on this master (check consistency between /etc/pki/pki-tomcat/alias, the certs in cn=certificates,cn=ipa,cn=etc,$BASEDN, and the content in /etc/pki/pki-tomcat/ca/CS.cfg).

Then check that replication is working between the renewal master and the other masters. If the replication is broken, the certs will not be copied on the other masters and the download will not detect new certificates.

HTH,
Flo
$ getcert list | grep -E "Request ID|status|certificate|expires"
Number of certificates and requests being tracked: 9.
Request ID '20170920090053':
     status: MONITORING
     certificate: type=FILE,location='/var/kerberos/krb5kdc/kdc.crt'
     expires: 2018-09-20 09:00:53 UTC
     certificate template/profile: KDCs_PKINIT_Certs
Request ID '20171221120303':
     status: CA_WORKING
    certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB'
     expires: 2018-03-27 14:07:51 UTC
Request ID '20171221120304':
     status: CA_WORKING
    certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS Certificate DB'
     expires: 2018-03-27 14:07:50 UTC
Request ID '20171221120305':
     status: CA_WORKING
    certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB'
     expires: 2018-03-27 14:07:51 UTC
Request ID '20171221120306':
     status: MONITORING
    certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca',token='NSS Certificate DB'
     expires: 2036-04-06 14:07:49 UTC
Request ID '20171221120307':
     status: CA_WORKING
     certificate: type=FILE,location='/var/lib/ipa/ra-agent.pem'
     expires: 2018-03-27 14:08:18 UTC
Request ID '20171221120308':
     status: MONITORING
    certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS Certificate DB'
     expires: 2019-07-25 16:13:23 UTC
Request ID '20171221120309':
     status: MONITORING
    certificate: type=NSSDB,location='/etc/dirsrv/slapd-PRIVATE-CCNR-CEB-PRIVATE-CAM-AC-UK',nickname='Server-Cert',token='NSS Certificate DB'
     expires: 2019-04-09 12:11:52 UTC
Request ID '20171221120310':
     status: MONITORING
    certificate: type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS Certificate DB'
     expires: 2019-04-09 12:11:54 UTC
_______________________________________________
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org
To unsubscribe send an email to freeipa-users-le...@lists.fedorahosted.org
_______________________________________________
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org
To unsubscribe send an email to freeipa-users-le...@lists.fedorahosted.org

Reply via email to