The best that I can determine from the stack traces that you've shown and the error messages is that Puppetserver somehow can't associate the CRLs that you've provided with your external CA or signed certificate. It looks like adding `-Djava.security.debug=all` will provide a lot of information into SSL operations - though it's incredibly verbose so you won't want to do this on a production node. I'm doing some tests on my end with this as well but you may want to test this in your environment and see if it provides any useful information.
Would it be possible for you to provide your CA cert/client cert/CRL? Or is that private? Lastly, Puppet and Puppetserver have two CRLs. One is '/etc/puppetlabs/puppet/ssl/ca/ca_crt.pem' and the other is '/etc/puppetlabs/puppet/ssl/crl.pem'. It's possible for these to get out of sync which can lead to all sorts of confusing behavior. Could you confirm that those CRLs match? On Tuesday, March 21, 2017 at 3:22:24 PM UTC-7, Steve Viola wrote: > > To clarify, I this is with puppetserver 2.7.2 from the puppetlabs yum > repo. Using the version of openssl in /opt/puppetlabs/puppet/bin/ provides > the same output as the system openssl > > On Tuesday, March 21, 2017 at 4:52:38 PM UTC-4, Steve Viola wrote: >> >> Hello, >> >> I've configured my puppetserver with an External CA >> <https://docs.puppet.com/puppetserver/latest/external_ca_configuration.html>, >> >> and everything was working as expected off the bat, but when I add the CRL >> path, puppet agent runs on all hosts stops running. webserver.conf looks >> like this: >> >> webserver: { >>> access-log-config: /etc/puppetlabs/puppetserver/request-logging.xml >>> client-auth: want >>> ssl-host: 0.0.0.0 >>> ssl-port: 8140 >>> ssl-cert: /etc/puppetlabs/puppet/ssl/certs/<hostname> >>> ssl-key: /etc/puppetlabs/puppet/ssl/private_keys/<hostname> >>> ssl-ca-cert: /etc/puppetlabs/puppet/ssl/ca/ca.crt >>> ssl-cert-chain: /etc/puppetlabs/puppet/ssl/ca/ca_chain.pem >>> ssl-crl-path: /etc/puppetlabs/puppet/ssl/ca/ca_crl.pem >>> } >> >> >> At first there were not any errors appearing in the puppetserver logs, >> but after changing the logback log level to DEBUG, I finally saw found >> errors in the puppetserver.log file: >> >> 2017-03-21 16:28:11,652 DEBUG [qtp1057116152-68] [o.e.j.s.HttpConnection] >>> javax.net.ssl.SSLHandshakeException: General SSLEngine problem >>> at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1478) >>> at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:535) >>> at sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:813) >>> at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:781) >>> at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624) >>> at >>> org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.fill(SslConnection.java:516) >>> at >>> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:239) >>> at >>> org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540) >>> at >>> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) >>> at >>> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) >>> at java.lang.Thread.run(Thread.java:745) >>> Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem >>> at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) >>> at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1728) >>> at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:304) >>> at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:296) >>> at >>> sun.security.ssl.ServerHandshaker.clientCertificate(ServerHandshaker.java:1906) >>> at >>> sun.security.ssl.ServerHandshaker.processMessage(ServerHandshaker.java:233) >>> at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1026) >>> at sun.security.ssl.Handshaker$1.run(Handshaker.java:966) >>> at sun.security.ssl.Handshaker$1.run(Handshaker.java:963) >>> at java.security.AccessController.doPrivileged(Native Method) >>> at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1416) >>> at >>> org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.fill(SslConnection.java:612) >>> ... 5 common frames omitted >>> Caused by: sun.security.validator.ValidatorException: PKIX path >>> validation failed: java.security.cert.CertPathValidatorException: Could not >>> determine revocation status >>> at >>> sun.security.validator.PKIXValidator.doValidate(PKIXValidator.java:352) >>> at >>> sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:249) >>> at sun.security.validator.Validator.validate(Validator.java:260) >>> at >>> sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324) >>> at >>> sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:279) >>> at >>> sun.security.ssl.X509TrustManagerImpl.checkClientTrusted(X509TrustManagerImpl.java:130) >>> at >>> sun.security.ssl.ServerHandshaker.clientCertificate(ServerHandshaker.java:1893) >>> ... 12 common frames omitted >>> Caused by: java.security.cert.CertPathValidatorException: Could not >>> determine revocation status >>> at >>> sun.security.provider.certpath.PKIXMasterCertPathValidator.validate(PKIXMasterCertPathValidator.java:135) >>> at >>> sun.security.provider.certpath.PKIXCertPathValidator.validate(PKIXCertPathValidator.java:219) >>> at >>> sun.security.provider.certpath.PKIXCertPathValidator.validate(PKIXCertPathValidator.java:140) >>> at >>> sun.security.provider.certpath.PKIXCertPathValidator.engineValidate(PKIXCertPathValidator.java:79) >>> at >>> java.security.cert.CertPathValidator.validate(CertPathValidator.java:292) >>> at >>> sun.security.validator.PKIXValidator.doValidate(PKIXValidator.java:347) >>> ... 18 common frames omitted >>> Caused by: java.security.cert.CertPathValidatorException: Could not >>> determine revocation status >>> at >>> sun.security.provider.certpath.RevocationChecker.buildToNewKey(RevocationChecker.java:1092) >>> at >>> sun.security.provider.certpath.RevocationChecker.verifyWithSeparateSigningKey(RevocationChecker.java:910) >>> at >>> sun.security.provider.certpath.RevocationChecker.checkCRLs(RevocationChecker.java:577) >>> at >>> sun.security.provider.certpath.RevocationChecker.checkCRLs(RevocationChecker.java:465) >>> at >>> sun.security.provider.certpath.RevocationChecker.check(RevocationChecker.java:367) >>> at >>> sun.security.provider.certpath.RevocationChecker.check(RevocationChecker.java:337) >>> at >>> sun.security.provider.certpath.PKIXMasterCertPathValidator.validate(PKIXMasterCertPathValidator.java:125) >>> ... 23 common frames omitted >> >> >> Testing the CRL using openssl works as expected, after concatenating the >> CA crt and the CRL crt, and running the openssl command below verifies the >> cert hasn't been revoked. >> >> $ openssl verify -crl_check -CAfile crl_ca.pem >>> /etc/puppetlabs/puppet/ssl/certs/<hostname>.pem >>> /etc/puppetlabs/puppet/ssl/certs/<hostname>.pem: OK >> >> >> OpenSSL also verifies that certs have been revoked as well: >> >> $ openssl verify -crl_check -CAfile crl_chain3.pem <revoked cert>.pem >>> <revoked cert>.pem: O = <domain>, CN = <hostname> >>> error 23 at 0 depth lookup:certificate revoked >> >> >> Are there any additional setting needed to get Java working to honor the >> CRL? Is there any resource for better logging to be able to narrow down the >> issue? Is using a CRL with an external CA still supported in puppetserver, >> or should I avoid using a CRL and use OCSP instead? >> >> Any help or advice would be hugely appreciated. >> >> Thanks a lot. >> > -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/ee6e88cb-a459-4c8f-808b-46d32dab1fa2%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
