On 08/01/2017 12:03 PM, Rob Crittenden wrote:
Ian Harding wrote:
On 08/01/2017 07:39 AM, Florence Blanc-Renaud wrote:
On 08/01/2017 03:11 PM, Ian Harding wrote:
On 08/01/2017 01:48 AM, Florence Blanc-Renaud wrote:
On 08/01/2017 01:32 AM, Ian Harding via FreeIPA-users wrote:

On 07/31/2017 11:34 AM, Rob Crittenden wrote:
Ian Harding via FreeIPA-users wrote:
I had an unexpected restart of an IPA server that had apparently had
updates run but had not been restarted.  ipactl says pki-tomcatd
not start.

Strangely, the actual service appears to be running:

dogtag is an application within tomcat so tomcat can run without

We need to see more of the dogtag debug log to see what is going on.

It looks like an authentication problem...

[28/Jul/2017:10:08:47][localhost-startStop-1]: SSL handshake happened
Could not connect to LDAP server host seattlenfs.bpt.rocks port 636
Error netscape.ldap.LDAPException: Authentication failed (49)


dogtag stores its internal data in the LDAP server and needs to
establish a secure LDAP connection. You can check how this
connection is configured in /etc/pki/pki-tomcat/ca/CS.cfg, look for
the lines:

internaldb.ldapauth.bindDN=cn=Directory Manager
internaldb.ldapauth.clientCertNickname=subsystemCert cert-pki-ca

authtype can be SslClientAuth (authentication with a ssl
certificate) or BasicAuth (authentication with a bind DN and
password stored in /var/lib/pki/pki-tomcat/conf/password.conf).

You can use this information to manually check the credentials. For
instance with sslclientauth:

export LDAPTLS_CACERTDIR=/etc/pki/pki-tomcat/alias
export LDAPTLS_CERT='subsystemCert cert-pki-ca'

ldapsearch -H ldaps://`hostname`:636 -b "" -s base -Y EXTERNAL
(provide the password from /etc/pki/pki-tomcat/alias/pwdfile.txt)

I found this:

internaldb.ldapauth.clientCertNickname=subsystemCert cert-pki-ca

and when I try the ldapsearch I am presented with a prompt to provide
a pin/password

Please enter pin, password, or pass phrase for security token 'ldap(0)':

but there is no password file...


you are right, in 4.4. there is no pwdfile.txt and the password can be
found in /var/lib/pki/pki-tomcat/conf/password.conf (with the tag

Can you check if the password with the tag internal=... allows to read
the keys from the NSS db?
certutil -K -d /etc/pki/pki-tomcat/alias
(provide password)

That works...

# certutil -K -d /etc/pki/pki-tomcat/alias
certutil: Checking token "NSS Certificate DB" in slot "NSS User Private
Key and Certificate Services"
Enter Password or Pin for "NSS Certificate DB":
< 0> rsa      0f327e760a7eecdcf6973f5dc57ca5367c592d64   (orphan)
< 1> rsa      b12580c7c696cfcd8aefc9405a7a870b24b7b96a   NSS Certificate
DB:auditSigningCert cert-pki-ca
< 2> rsa      881b7254c40fa40bc50681bcc8d37bb3eb49937e   caSigningCert
< 3> rsa      fa9a255a1d15585ac28064c0f4986e416bc48403   NSS Certificate
DB:ocspSigningCert cert-pki-ca
< 4> rsa      3fb609d0f7d72c2d325d6a2dc16577a7f7e5a01f   Server-Cert
< 5> rsa      1e9479a9556af9339bb5e4552ccbd381d3c38856   NSS Certificate
DB:subsystemCert cert-pki-ca

But this doesn't (with the same password from password.conf)

# ldapsearch -H ldaps://`hostname`:636 -b "" -s base -Y EXTERNAL
Please enter pin, password, or pass phrase for security token 'ldap(0)':
SASL/EXTERNAL authentication started
ldap_sasl_interactive_bind_s: Invalid credentials (49)

That password is getting me somewhere though, since if I put in a
nonsense or incorrect password it just prompts over and over.

Let's step back a second. You upgraded from what to what?

There wasn't much of a change... I just assumed someone ran yum upgrade and didn't restart, then the power outage... it looks like not much of a version change though.

# grep ipa /var/log/yum.log
Jan 08 04:45:32 Installed: ipa-common-4.4.0-14.el7.centos.1.1.noarch
Jan 08 04:45:32 Installed: ipa-client-common-4.4.0-14.el7.centos.1.1.noarch
Jan 08 04:46:06 Updated: libipa_hbac-1.14.0-43.el7_3.4.x86_64
Jan 08 04:46:07 Updated: python-libipa_hbac-1.14.0-43.el7_3.4.x86_64
Jan 08 04:46:08 Installed: python-ipaddress-1.0.16-2.el7.noarch
Jan 08 04:47:04 Installed: python2-ipalib-4.4.0-14.el7.centos.1.1.noarch
Jan 08 04:47:05 Installed: python2-ipaclient-4.4.0-14.el7.centos.1.1.noarch
Jan 08 04:47:05 Updated: ipa-admintools-4.4.0-14.el7.centos.1.1.noarch
Jan 08 04:47:05 Installed: ipa-server-common-4.4.0-14.el7.centos.1.1.noarch
Jan 08 04:47:06 Installed: python2-ipaserver-4.4.0-14.el7.centos.1.1.noarch
Jan 08 04:47:07 Updated: sssd-ipa-1.14.0-43.el7_3.4.x86_64
Jan 08 04:47:17 Updated: ipa-client-4.4.0-14.el7.centos.1.1.x86_64
Jan 08 04:47:23 Updated: ipa-server-4.4.0-14.el7.centos.1.1.x86_64
Jan 08 04:47:27 Updated: ipa-server-dns-4.4.0-14.el7.centos.1.1.noarch
Jan 08 04:47:36 Installed: ipa-python-compat-4.4.0-14.el7.centos.1.1.noarch
Jan 08 04:50:07 Erased: ipa-python-4.2.0-15.0.1.el7.centos.19.x86_64
Jul 28 09:55:41 Updated: ipa-common-4.4.0-14.el7.centos.7.noarch
Jul 28 09:55:42 Updated: ipa-client-common-4.4.0-14.el7.centos.7.noarch
Jul 28 09:55:43 Updated: ipa-server-common-4.4.0-14.el7.centos.7.noarch
Jul 28 09:55:43 Updated: python2-ipalib-4.4.0-14.el7.centos.7.noarch
Jul 28 09:55:44 Updated: python2-ipaclient-4.4.0-14.el7.centos.7.noarch
Jul 28 09:55:45 Updated: python2-ipaserver-4.4.0-14.el7.centos.7.noarch
Jul 28 09:55:45 Updated: ipa-admintools-4.4.0-14.el7.centos.7.noarch
Jul 28 09:55:46 Updated: ipa-client-4.4.0-14.el7.centos.7.x86_64
Jul 28 09:55:48 Updated: ipa-server-4.4.0-14.el7.centos.7.x86_64
Jul 28 09:55:48 Updated: ipa-server-dns-4.4.0-14.el7.centos.7.noarch
Jul 28 09:55:48 Updated: ipa-python-compat-4.4.0-14.el7.centos.7.noarch

# grep pki /var/log/yum.log
Jan 08 04:46:05 Updated: pki-base-10.3.3-14.el7_3.noarch
Jan 08 04:46:09 Updated: krb5-pkinit-1.14.1-27.el7_3.x86_64
Jan 08 04:47:19 Installed: pki-base-java-10.3.3-14.el7_3.noarch
Jan 08 04:47:19 Updated: pki-tools-10.3.3-14.el7_3.x86_64
Jan 08 04:47:21 Updated: pki-server-10.3.3-14.el7_3.noarch
Jan 08 04:47:22 Updated: pki-kra-10.3.3-14.el7_3.noarch
Jan 08 04:47:22 Updated: pki-ca-10.3.3-14.el7_3.noarch
Jul 28 10:13:16 Updated: pki-base-10.3.3-19.el7_3.noarch
Jul 28 10:13:17 Updated: pki-base-java-10.3.3-19.el7_3.noarch
Jul 28 10:13:17 Updated: pki-tools-10.3.3-19.el7_3.x86_64
Jul 28 10:13:21 Updated: pki-server-10.3.3-19.el7_3.noarch
Jul 28 10:13:21 Updated: pki-kra-10.3.3-19.el7_3.noarch
Jul 28 10:13:21 Updated: pki-ca-10.3.3-19.el7_3.noarch

Are you running in SELinux enforcing mode?

# getenforce

If so can you run:

# ausearch -m AVC

# ausearch -m AVC
The NOLOG option to log_format is deprecated. Please use the write_logs option.
The NOLOG option is overriding the write_logs current setting.
time->Mon Mar 14 10:39:01 2016
type=SYSCALL msg=audit(1457977141.767:17537): arch=c000003e syscall=2 success=no exit=-13 a0=7fbcab038d10 a1=800 a2=1 a3=7fbca60f42e0 items=0 ppid=1406 pid=12965 auid=4294967295 uid=0 gid=0 euid=581400001 suid=0 fsuid=581400001 egid=581400001 sgid=0 fsgid=581400001 tty=(none) ses=4294967295 comm="sshd" exe="/usr/sbin/sshd" subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 key=(null) type=AVC msg=audit(1457977141.767:17537): avc: denied { read } for pid=12965 comm="sshd" name="authorized_keys" dev="dm-2" ino=116393517 scontext=system_u:system_r:sshd_t:s0-s0:c0.c1023 tcontext=system_u:object_r:unlabeled_t:s0 tclass=file

I suspect that the subsystem cert was renewed and everything wasn't
updated properly. If I'm right this is unrelated to the upgrade it just
shone a spotlight on it.

Can you run these commands:

$ ldapsearch -LLL -D 'cn=directory manager' -W -b
uid=pkidbuser,ou=people,o=ipaca userCertificate description

This returns

$ ldapsearch -LLL -D 'cn=directory manager' -W -b uid=pkidbuser,ou=people,o=ipaca userCertificate description
Enter LDAP Password:
dn: uid=pkidbuser,ou=people,o=ipaca
... stuff ...
description: 2;4;CN=Certificate Authority,O=BPT.ROCKS;CN=CA Subsystem,O=BPT.ROCKS


# certutil -L -d /etc/pki/pki-tomcat/alias -n 'subsystemCert
cert-pki-ca' | grep Serial
# certutil -L -d /etc/pki/pki-tomcat/alias -n 'subsystemCert cert-pki-ca' -a

These both return

# certutil -K -d /etc/pki/pki-tomcat/alias -n 'subsystemCert cert-pki-ca' | grep Serial
Enter Password or Pin for "NSS Certificate DB":
certutil: problem listing keys: SEC_ERROR_UNRECOGNIZED_OID: Unrecognized Object Identifier.

# certutil -K -d /etc/pki/pki-tomcat/alias -n 'subsystemCert cert-pki-ca' -a certutil: Checking token "NSS Certificate DB" in slot "NSS User Private Key and Certificate Services"
Enter Password or Pin for "NSS Certificate DB":
certutil: problem listing keys: SEC_ERROR_UNRECOGNIZED_OID: Unrecognized Object Identifier.

which seems weird.

The description attribute in the ldapsearch output is of the form 2;#;DN;DN

The # should match the serial number in the first certutil output

The ASCII blob (minus the BEGIN/END headers) should match one of the
userCertificate attributes.


Ian Harding
IT Director
Brown Paper Tickets
1-800-838-3006 ext 7186
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org
To unsubscribe send an email to freeipa-users-le...@lists.fedorahosted.org

Reply via email to