Re: [Freeipa-users] What happened to my {cacert,kdc}.pem files?

2013-07-09 Thread Rob Crittenden

Brian Vetter wrote:

We had to shut down our FREEIPA server and move it. When I brought it back up 
again today (all same IPs, network, etc), it failed to come up. I see lots of  
various forms of the following messages when trying to start the ipa, named, 
and other services:


What do you mean by move it? Physically move a machine or did you try to 
move the configuration?


rob



Failed to init credentials (Cannot contact any KDC for realm ...
startup - The default password storage scheme SSHA could not be read or was not 
found in the file /etc/dirsrv/slapd-TESTREALM.COM/dse.ldif. It is mandatory.
startup - The default password storage scheme SSHA could not be read or was not 
found in the file /etc/dirsrv/slapd-PKI-IPA/dse.ldif. It is mandatory.
krb5kdc: Server error - while fetching master key K/M for realm TESTREALM.COM
kinit: Cannot contact any KDC for realm 'TESTREALM.COM' while getting initial 
credentials


From what I can surmise after seeing these, something in kerberos is messed up. 
I don't know for sure if it is related, but I see that the files referenced in 
/var/kerberos/krb5kdc/kdc.conf are not there. In particular,


pkinit_identity = FILE:/var/kerberos/krb5kdc/kdc.pem
pkinit_anchors = FILE:/var/kerberos/krb5kdc/cacert.pem

If this is likely the case (or perhaps just the first thing I've run into that is wrong), how do I 
go about recovering them? I've tried (with fingers crossed) yum reinstall 
freeipa-server and yum update freeipa-server hoping that they'd see the need to 
fix this. They didn't. Still get the same errors.

Is there some backdoor way to recreate these files from elsewhere in the 
install? Perhaps buried in the 389 directory server's database and accessible 
using db4.4_dump or some other tools? If there is no way to recreate them, is 
there a way to reassert new keys without having to start all over? And if I 
have to start all over, is there anyway to extract some of the records from the 
dir DB so I can reload them with a new server?

Thanks for any suggestions/guidance,

Brian


___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users



___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users


Re: [Freeipa-users] What happened to my {cacert,kdc}.pem files?

2013-07-09 Thread Brian Vetter
Here is the directory listing ...

On Jul 8, 2013, at 8:13 PM, Rich Megginson wrote:

 On 07/08/2013 06:15 PM, Brian Vetter wrote:
 We had to shut down our FREEIPA server and move it. When I brought it back 
 up again today (all same IPs, network, etc), it failed to come up. I see 
 lots of  various forms of the following messages when trying to start the 
 ipa, named, and other services:
 
 Failed to init credentials (Cannot contact any KDC for realm ...
 startup - The default password storage scheme SSHA could not be read or was 
 not found in the file /etc/dirsrv/slapd-TESTREALM.COM/dse.ldif. It is 
 mandatory.
 startup - The default password storage scheme SSHA could not be read or was 
 not found in the file /etc/dirsrv/slapd-PKI-IPA/dse.ldif. It is mandatory.
 
 ls -alrtF /etc/dirsrv/slapd-*

# ls -alrtF /etc/dirsrv/slapd-*
/etc/dirsrv/slapd-PKI-IPA:
total 484
-r--r-. 1 pkisrv dirsrv  33763 Sep 25  2012 dse_original.ldif
-r--r-. 1 pkisrv dirsrv   3595 Sep 25  2012 certmap.conf
-r--r-. 1 pkisrv dirsrv   5366 Sep 25  2012 slapd-collations.conf
-rw-rw. 1 pkisrv dirsrv  16384 Sep 25  2012 secmod.db.orig
-rw---. 1 pkisrv dirsrv 40 Sep 25  2012 pwdfile.txt
-r. 1 pkisrv dirsrv 66 Sep 25  2012 pin.txt
drwxrwxr-x. 6 root   dirsrv   4096 Sep 25  2012 ../
-rw-rw. 1 pkisrv dirsrv  16384 Sep 25  2012 key3.db.orig
-rw-rw. 1 pkisrv dirsrv  65536 Sep 25  2012 cert8.db.orig
-rw---. 1 pkisrv dirsrv 111599 Jun 24 15:33 dse.ldif.startOK
drwxrwx---. 2 pkisrv dirsrv   4096 Jun 24 15:33 schema/
-rw---. 1 pkisrv root16384 Jun 24 15:33 secmod.db
-rw---. 1 pkisrv dirsrv 111599 Jun 24 15:33 dse.ldif.bak
-rw---. 1 pkisrv dirsrv  0 Jul  3 18:43 dse.ldif
drwxrwx---. 3 pkisrv dirsrv   4096 Jul  3 18:43 ./
-rw---. 1 pkisrv root16384 Jul  8 21:31 key3.db
-rw---. 1 pkisrv root65536 Jul  8 21:31 cert8.db

/etc/dirsrv/slapd-TESTREALM-COM:
total 1316
-r--r-. 1 dirsrv dirsrv 33866 Sep 25  2012 dse_original.ldif
-r--r-. 1 dirsrv dirsrv  5366 Sep 25  2012 slapd-collations.conf
-rw-rw. 1 dirsrv dirsrv 16384 Sep 25  2012 secmod.db.orig
-rw---. 1 dirsrv dirsrv40 Sep 25  2012 pwdfile.txt
-r. 1 dirsrv dirsrv66 Sep 25  2012 pin.txt
-r--r-. 1 dirsrv dirsrv  3637 Sep 25  2012 certmap.conf
-rw-rw. 1 dirsrv dirsrv 16384 Sep 25  2012 key3.db.orig
-rw-rw. 1 dirsrv dirsrv 65536 Sep 25  2012 cert8.db.orig
drwxrwxr-x. 6 root   dirsrv  4096 Sep 25  2012 ../
-rw---. 1 dirsrv root   88102 Oct 16  2012 dse.ldif.ipa.7536ea943b6ffd19
-rw---. 1 dirsrv root   88050 Oct 18  2012 dse.ldif.ipa.b321343f4245e859
-rw---. 1 dirsrv root   88050 Oct 28  2012 dse.ldif.ipa.6f187ed275f2c8d6
-rw---. 1 dirsrv root   88050 Oct 31  2012 dse.ldif.ipa.a77259fe47a3f1ef
-rw---. 1 dirsrv root   88050 Dec  5  2012 dse.ldif.ipa.45e94baeae26de8b
-rw---. 1 dirsrv root   88050 Dec  5  2012 dse.ldif.ipa.df63ce99558b2b8b
-rw---. 1 dirsrv root   88361 Dec 19  2012 dse.ldif.ipa.2808d9c2613eaf22
-rw---. 1 dirsrv root   88361 Jan 21 14:22 dse.ldif.ipa.da912fc817573d85
-rw---. 1 dirsrv root   88361 Mar 16 14:03 dse.ldif.ipa.17df93a6a8d16ed9
-rw---. 1 dirsrv root   88361 Jun 24 15:33 dse.ldif.ipa.f5dec6078ee62fe5
-rw---. 1 dirsrv dirsrv 88359 Jun 24 15:33 dse.ldif.startOK
drwxrwx---. 2 dirsrv dirsrv  4096 Jun 24 15:33 schema/
-rw---. 1 dirsrv root   16384 Jun 24 15:33 secmod.db
-rw---. 1 dirsrv dirsrv 88361 Jun 24 15:33 dse.ldif.bak
-rw---. 1 root   root   0 Jul  3 18:43 dse.ldif.ipa.e9532be9acc9603f
-rw---. 1 root   root   0 Jul  3 18:43 dse.ldif.ipa.5cec24995ad13b30
-rw---. 1 dirsrv dirsrv 0 Jul  3 18:43 dse.ldif
drwxrwx---. 3 dirsrv dirsrv  4096 Jul  8 18:50 ./
-rw---. 1 dirsrv root   16384 Jul  8 21:31 key3.db
-rw---. 1 dirsrv root   65536 Jul  8 21:31 cert8.db

 
 krb5kdc: Server error - while fetching master key K/M for realm 
 TESTREALM.COM
 kinit: Cannot contact any KDC for realm 'TESTREALM.COM' while getting 
 initial credentials
 
 From what I can surmise after seeing these, something in kerberos is messed 
 up. I don't know for sure if it is related, but I see that the files 
 referenced in /var/kerberos/krb5kdc/kdc.conf are not there. In particular,
 
 pkinit_identity = FILE:/var/kerberos/krb5kdc/kdc.pem
 pkinit_anchors = FILE:/var/kerberos/krb5kdc/cacert.pem
 
 If this is likely the case (or perhaps just the first thing I've run into 
 that is wrong), how do I go about recovering them? I've tried (with fingers 
 crossed) yum reinstall freeipa-server and yum update freeipa-server 
 hoping that they'd see the need to fix this. They didn't. Still get the same 
 errors.
 
 Is there some backdoor way to recreate these files from elsewhere in the 
 install? Perhaps buried in the 389 directory server's database and 
 accessible using db4.4_dump or some other tools? If there is no way to 
 recreate them, is there a way to reassert new keys without having to start 
 all over? And if 

Re: [Freeipa-users] What happened to my {cacert,kdc}.pem files?

2013-07-09 Thread Rich Megginson

On 07/09/2013 12:49 PM, Brian Vetter wrote:

Here is the directory listing ...

On Jul 8, 2013, at 8:13 PM, Rich Megginson wrote:


On 07/08/2013 06:15 PM, Brian Vetter wrote:

We had to shut down our FREEIPA server and move it. When I brought it back up 
again today (all same IPs, network, etc), it failed to come up. I see lots of  
various forms of the following messages when trying to start the ipa, named, 
and other services:

Failed to init credentials (Cannot contact any KDC for realm ...
startup - The default password storage scheme SSHA could not be read or was not 
found in the file /etc/dirsrv/slapd-TESTREALM.COM/dse.ldif. It is mandatory.
startup - The default password storage scheme SSHA could not be read or was not 
found in the file /etc/dirsrv/slapd-PKI-IPA/dse.ldif. It is mandatory.

ls -alrtF /etc/dirsrv/slapd-*

# ls -alrtF /etc/dirsrv/slapd-*
/etc/dirsrv/slapd-PKI-IPA:
total 484
-r--r-. 1 pkisrv dirsrv  33763 Sep 25  2012 dse_original.ldif
-r--r-. 1 pkisrv dirsrv   3595 Sep 25  2012 certmap.conf
-r--r-. 1 pkisrv dirsrv   5366 Sep 25  2012 slapd-collations.conf
-rw-rw. 1 pkisrv dirsrv  16384 Sep 25  2012 secmod.db.orig
-rw---. 1 pkisrv dirsrv 40 Sep 25  2012 pwdfile.txt
-r. 1 pkisrv dirsrv 66 Sep 25  2012 pin.txt
drwxrwxr-x. 6 root   dirsrv   4096 Sep 25  2012 ../
-rw-rw. 1 pkisrv dirsrv  16384 Sep 25  2012 key3.db.orig
-rw-rw. 1 pkisrv dirsrv  65536 Sep 25  2012 cert8.db.orig
-rw---. 1 pkisrv dirsrv 111599 Jun 24 15:33 dse.ldif.startOK
drwxrwx---. 2 pkisrv dirsrv   4096 Jun 24 15:33 schema/
-rw---. 1 pkisrv root16384 Jun 24 15:33 secmod.db
-rw---. 1 pkisrv dirsrv 111599 Jun 24 15:33 dse.ldif.bak
-rw---. 1 pkisrv dirsrv  0 Jul  3 18:43 dse.ldif
drwxrwx---. 3 pkisrv dirsrv   4096 Jul  3 18:43 ./
-rw---. 1 pkisrv root16384 Jul  8 21:31 key3.db
-rw---. 1 pkisrv root65536 Jul  8 21:31 cert8.db

/etc/dirsrv/slapd-TESTREALM-COM:
total 1316
-r--r-. 1 dirsrv dirsrv 33866 Sep 25  2012 dse_original.ldif
-r--r-. 1 dirsrv dirsrv  5366 Sep 25  2012 slapd-collations.conf
-rw-rw. 1 dirsrv dirsrv 16384 Sep 25  2012 secmod.db.orig
-rw---. 1 dirsrv dirsrv40 Sep 25  2012 pwdfile.txt
-r. 1 dirsrv dirsrv66 Sep 25  2012 pin.txt
-r--r-. 1 dirsrv dirsrv  3637 Sep 25  2012 certmap.conf
-rw-rw. 1 dirsrv dirsrv 16384 Sep 25  2012 key3.db.orig
-rw-rw. 1 dirsrv dirsrv 65536 Sep 25  2012 cert8.db.orig
drwxrwxr-x. 6 root   dirsrv  4096 Sep 25  2012 ../
-rw---. 1 dirsrv root   88102 Oct 16  2012 dse.ldif.ipa.7536ea943b6ffd19
-rw---. 1 dirsrv root   88050 Oct 18  2012 dse.ldif.ipa.b321343f4245e859
-rw---. 1 dirsrv root   88050 Oct 28  2012 dse.ldif.ipa.6f187ed275f2c8d6
-rw---. 1 dirsrv root   88050 Oct 31  2012 dse.ldif.ipa.a77259fe47a3f1ef
-rw---. 1 dirsrv root   88050 Dec  5  2012 dse.ldif.ipa.45e94baeae26de8b
-rw---. 1 dirsrv root   88050 Dec  5  2012 dse.ldif.ipa.df63ce99558b2b8b
-rw---. 1 dirsrv root   88361 Dec 19  2012 dse.ldif.ipa.2808d9c2613eaf22
-rw---. 1 dirsrv root   88361 Jan 21 14:22 dse.ldif.ipa.da912fc817573d85
-rw---. 1 dirsrv root   88361 Mar 16 14:03 dse.ldif.ipa.17df93a6a8d16ed9
-rw---. 1 dirsrv root   88361 Jun 24 15:33 dse.ldif.ipa.f5dec6078ee62fe5
-rw---. 1 dirsrv dirsrv 88359 Jun 24 15:33 dse.ldif.startOK
drwxrwx---. 2 dirsrv dirsrv  4096 Jun 24 15:33 schema/
-rw---. 1 dirsrv root   16384 Jun 24 15:33 secmod.db
-rw---. 1 dirsrv dirsrv 88361 Jun 24 15:33 dse.ldif.bak
-rw---. 1 root   root   0 Jul  3 18:43 dse.ldif.ipa.e9532be9acc9603f
-rw---. 1 root   root   0 Jul  3 18:43 dse.ldif.ipa.5cec24995ad13b30
-rw---. 1 dirsrv dirsrv 0 Jul  3 18:43 dse.ldif
drwxrwx---. 3 dirsrv dirsrv  4096 Jul  8 18:50 ./
-rw---. 1 dirsrv root   16384 Jul  8 21:31 key3.db
-rw---. 1 dirsrv root   65536 Jul  8 21:31 cert8.db


if 389/dirsrv is not running, you can replace the 0 length dse.ldif with 
the dse.ldif.bak.

cp -p dse.ldif.bak dse.ldif

We have fixed this issue in 1.3.2

Are these servers running in a VM?



krb5kdc: Server error - while fetching master key K/M for realm TESTREALM.COM
kinit: Cannot contact any KDC for realm 'TESTREALM.COM' while getting initial 
credentials

From what I can surmise after seeing these, something in kerberos is messed 
up. I don't know for sure if it is related, but I see that the files referenced in 
/var/kerberos/krb5kdc/kdc.conf are not there. In particular,

pkinit_identity = FILE:/var/kerberos/krb5kdc/kdc.pem
pkinit_anchors = FILE:/var/kerberos/krb5kdc/cacert.pem

If this is likely the case (or perhaps just the first thing I've run into that is wrong), how do I 
go about recovering them? I've tried (with fingers crossed) yum reinstall 
freeipa-server and yum update freeipa-server hoping that they'd see the need to 
fix this. They didn't. Still get the same errors.

Is there some backdoor way to recreate these files from elsewhere in the 
install? Perhaps buried in the 

Re: [Freeipa-users] What happened to my {cacert,kdc}.pem files?

2013-07-09 Thread Rich Megginson

On 07/09/2013 01:08 PM, Brian Vetter wrote:

Copying dse.ldif.bak worked.


Great!  Are these systems running on a VM?



Thanks,

Brian

On Jul 9, 2013, at 1:53 PM, Rich Megginson wrote:


On 07/09/2013 12:49 PM, Brian Vetter wrote:

Here is the directory listing ...

On Jul 8, 2013, at 8:13 PM, Rich Megginson wrote:


On 07/08/2013 06:15 PM, Brian Vetter wrote:

We had to shut down our FREEIPA server and move it. When I brought it back up 
again today (all same IPs, network, etc), it failed to come up. I see lots of  
various forms of the following messages when trying to start the ipa, named, 
and other services:

Failed to init credentials (Cannot contact any KDC for realm ...
startup - The default password storage scheme SSHA could not be read or was not 
found in the file /etc/dirsrv/slapd-TESTREALM.COM/dse.ldif. It is mandatory.
startup - The default password storage scheme SSHA could not be read or was not 
found in the file /etc/dirsrv/slapd-PKI-IPA/dse.ldif. It is mandatory.

ls -alrtF /etc/dirsrv/slapd-*

# ls -alrtF /etc/dirsrv/slapd-*
/etc/dirsrv/slapd-PKI-IPA:
total 484
-r--r-. 1 pkisrv dirsrv  33763 Sep 25  2012 dse_original.ldif
-r--r-. 1 pkisrv dirsrv   3595 Sep 25  2012 certmap.conf
-r--r-. 1 pkisrv dirsrv   5366 Sep 25  2012 slapd-collations.conf
-rw-rw. 1 pkisrv dirsrv  16384 Sep 25  2012 secmod.db.orig
-rw---. 1 pkisrv dirsrv 40 Sep 25  2012 pwdfile.txt
-r. 1 pkisrv dirsrv 66 Sep 25  2012 pin.txt
drwxrwxr-x. 6 root   dirsrv   4096 Sep 25  2012 ../
-rw-rw. 1 pkisrv dirsrv  16384 Sep 25  2012 key3.db.orig
-rw-rw. 1 pkisrv dirsrv  65536 Sep 25  2012 cert8.db.orig
-rw---. 1 pkisrv dirsrv 111599 Jun 24 15:33 dse.ldif.startOK
drwxrwx---. 2 pkisrv dirsrv   4096 Jun 24 15:33 schema/
-rw---. 1 pkisrv root16384 Jun 24 15:33 secmod.db
-rw---. 1 pkisrv dirsrv 111599 Jun 24 15:33 dse.ldif.bak
-rw---. 1 pkisrv dirsrv  0 Jul  3 18:43 dse.ldif
drwxrwx---. 3 pkisrv dirsrv   4096 Jul  3 18:43 ./
-rw---. 1 pkisrv root16384 Jul  8 21:31 key3.db
-rw---. 1 pkisrv root65536 Jul  8 21:31 cert8.db

/etc/dirsrv/slapd-TESTREALM-COM:
total 1316
-r--r-. 1 dirsrv dirsrv 33866 Sep 25  2012 dse_original.ldif
-r--r-. 1 dirsrv dirsrv  5366 Sep 25  2012 slapd-collations.conf
-rw-rw. 1 dirsrv dirsrv 16384 Sep 25  2012 secmod.db.orig
-rw---. 1 dirsrv dirsrv40 Sep 25  2012 pwdfile.txt
-r. 1 dirsrv dirsrv66 Sep 25  2012 pin.txt
-r--r-. 1 dirsrv dirsrv  3637 Sep 25  2012 certmap.conf
-rw-rw. 1 dirsrv dirsrv 16384 Sep 25  2012 key3.db.orig
-rw-rw. 1 dirsrv dirsrv 65536 Sep 25  2012 cert8.db.orig
drwxrwxr-x. 6 root   dirsrv  4096 Sep 25  2012 ../
-rw---. 1 dirsrv root   88102 Oct 16  2012 dse.ldif.ipa.7536ea943b6ffd19
-rw---. 1 dirsrv root   88050 Oct 18  2012 dse.ldif.ipa.b321343f4245e859
-rw---. 1 dirsrv root   88050 Oct 28  2012 dse.ldif.ipa.6f187ed275f2c8d6
-rw---. 1 dirsrv root   88050 Oct 31  2012 dse.ldif.ipa.a77259fe47a3f1ef
-rw---. 1 dirsrv root   88050 Dec  5  2012 dse.ldif.ipa.45e94baeae26de8b
-rw---. 1 dirsrv root   88050 Dec  5  2012 dse.ldif.ipa.df63ce99558b2b8b
-rw---. 1 dirsrv root   88361 Dec 19  2012 dse.ldif.ipa.2808d9c2613eaf22
-rw---. 1 dirsrv root   88361 Jan 21 14:22 dse.ldif.ipa.da912fc817573d85
-rw---. 1 dirsrv root   88361 Mar 16 14:03 dse.ldif.ipa.17df93a6a8d16ed9
-rw---. 1 dirsrv root   88361 Jun 24 15:33 dse.ldif.ipa.f5dec6078ee62fe5
-rw---. 1 dirsrv dirsrv 88359 Jun 24 15:33 dse.ldif.startOK
drwxrwx---. 2 dirsrv dirsrv  4096 Jun 24 15:33 schema/
-rw---. 1 dirsrv root   16384 Jun 24 15:33 secmod.db
-rw---. 1 dirsrv dirsrv 88361 Jun 24 15:33 dse.ldif.bak
-rw---. 1 root   root   0 Jul  3 18:43 dse.ldif.ipa.e9532be9acc9603f
-rw---. 1 root   root   0 Jul  3 18:43 dse.ldif.ipa.5cec24995ad13b30
-rw---. 1 dirsrv dirsrv 0 Jul  3 18:43 dse.ldif
drwxrwx---. 3 dirsrv dirsrv  4096 Jul  8 18:50 ./
-rw---. 1 dirsrv root   16384 Jul  8 21:31 key3.db
-rw---. 1 dirsrv root   65536 Jul  8 21:31 cert8.db

if 389/dirsrv is not running, you can replace the 0 length dse.ldif with the 
dse.ldif.bak.
cp -p dse.ldif.bak dse.ldif

We have fixed this issue in 1.3.2

Are these servers running in a VM?

krb5kdc: Server error - while fetching master key K/M for realm TESTREALM.COM
kinit: Cannot contact any KDC for realm 'TESTREALM.COM' while getting initial 
credentials

From what I can surmise after seeing these, something in kerberos is messed 
up. I don't know for sure if it is related, but I see that the files referenced in 
/var/kerberos/krb5kdc/kdc.conf are not there. In particular,

pkinit_identity = FILE:/var/kerberos/krb5kdc/kdc.pem
pkinit_anchors = FILE:/var/kerberos/krb5kdc/cacert.pem

If this is likely the case (or perhaps just the first thing I've run into that is wrong), how do I 
go about recovering them? I've tried (with fingers crossed) yum reinstall 
freeipa-server and yum update freeipa-server