Re: [Freeipa-users] replicas removed, but incorrectly

2016-09-26 Thread Ludwig Krispenz 

you should only remove agreements to no longer existing servers, eg where:

nsDS5ReplicaHost: kdc01.unix.iriszorg.nl

the other one should remain, not sure why it cannot contact the server

On 09/26/2016 03:35 PM, Natxo Asenjo wrote:

hi,

or do I need to remove:

dn: 
cn=cloneAgreement1-kdc03.unix.iriszorg.nl-pki-tomcat,cn=replica,cn=o\3Dipa

 ca,cn=mapping tree,cn=config

because it has this:
nsds5replicaLastUpdateStatus: -1 Unable to acquire replicaLDAP error: 
Can't co

 ntact LDAP server
nsds5replicaUpdateInProgress: FALSE

and this:

dn: 
cn=masterAgreement1-kdc04.unix.iriszorg.nl-pki-tomcat,cn=replica,cn=o\3Dip

 aca,cn=mapping tree,cn=config

nsds5replicaLastUpdateStatus: -1 Incremental update has failed and 
requires ad

 ministrator actionLDAP error: Can't contact LDAP server



On Mon, Sep 26, 2016 at 3:32 PM, Natxo Asenjo > wrote:


hi,



On Mon, Sep 26, 2016 at 3:06 PM, Ludwig Krispenz
> wrote:


On 09/26/2016 02:56 PM, Natxo Asenjo wrote:


so the command has not been successful in the kdc03. in the
dirsrv errors log  I see:

[26/Sep/2016:14:50:54 +0200] NSMMReplicationPlugin -
CleanAllRUV Task (rid 71): Not all replicas online, retrying
in 640 seconds...

this looks like there is still a replication agreement to one
of the no longer existing servers.

can you search for "... -b "cn=config"
"objectclass=nsds5replicationagreement"

and remove the ones no longer needed.


allow me to post the output of both commands as separate files

I am not really sure which one I need to remove.

--
Groeten,
natxo




--
--
Groeten,
natxo




--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander

-- 
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

Re: [Freeipa-users] replicas removed, but incorrectly

2016-09-26 Thread Natxo Asenjo 
hi,

or do I need to remove:

dn:
cn=cloneAgreement1-kdc03.unix.iriszorg.nl-pki-tomcat,cn=replica,cn=o\3Dipa
 ca,cn=mapping tree,cn=config

because it has this:
nsds5replicaLastUpdateStatus: -1 Unable to acquire replicaLDAP error: Can't
co
 ntact LDAP server
nsds5replicaUpdateInProgress: FALSE

and this:

dn:
cn=masterAgreement1-kdc04.unix.iriszorg.nl-pki-tomcat,cn=replica,cn=o\3Dip
 aca,cn=mapping tree,cn=config

nsds5replicaLastUpdateStatus: -1 Incremental update has failed and requires
ad
 ministrator actionLDAP error: Can't contact LDAP server



On Mon, Sep 26, 2016 at 3:32 PM, Natxo Asenjo 
wrote:

> hi,
>
>
>
> On Mon, Sep 26, 2016 at 3:06 PM, Ludwig Krispenz 
> wrote:
>
>>
>> On 09/26/2016 02:56 PM, Natxo Asenjo wrote:
>>
>>
>> so the command has not been successful in the kdc03. in the dirsrv errors
>> log  I see:
>>
>> [26/Sep/2016:14:50:54 +0200] NSMMReplicationPlugin - CleanAllRUV Task
>> (rid 71): Not all replicas online, retrying in 640 seconds...
>>
>> this looks like there is still a replication agreement to one of the no
>> longer existing servers.
>>
>> can you search for "... -b "cn=config" 
>> "objectclass=nsds5replicationagreement"
>>
>>
>> and remove the ones no longer needed.
>>
>
> allow me to post the output of both commands as separate files
>
> I am not really sure which one I need to remove.
>
> --
> Groeten,
> natxo
>



-- 
--
Groeten,
natxo
-- 
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

Re: [Freeipa-users] replicas removed, but incorrectly

2016-09-26 Thread Natxo Asenjo 
hi,



On Mon, Sep 26, 2016 at 3:06 PM, Ludwig Krispenz 
wrote:

>
> On 09/26/2016 02:56 PM, Natxo Asenjo wrote:
>
>
> so the command has not been successful in the kdc03. in the dirsrv errors
> log  I see:
>
> [26/Sep/2016:14:50:54 +0200] NSMMReplicationPlugin - CleanAllRUV Task (rid
> 71): Not all replicas online, retrying in 640 seconds...
>
> this looks like there is still a replication agreement to one of the no
> longer existing servers.
>
> can you search for "... -b "cn=config" "objectclass=nsds5replicationagreement"
>
>
> and remove the ones no longer needed.
>

allow me to post the output of both commands as separate files

I am not really sure which one I need to remove.

--
Groeten,
natxo
$ ldapsearch -Z -h kdc03.unix.iriszorg.nl -D "cn=Directory Manager" -W -b "cn=config" "objectclass=nsds5replicationagreement" 
Enter LDAP Password: 
# extended LDIF
#
# LDAPv3
# base

Re: [Freeipa-users] replicas removed, but incorrectly

2016-09-26 Thread Ludwig Krispenz 


On 09/26/2016 02:56 PM, Natxo Asenjo wrote:



On Mon, Sep 26, 2016 at 1:54 PM, Natxo Asenjo > wrote:





On Mon, Sep 26, 2016 at 1:50 PM, Ludwig Krispenz
> wrote:


On 09/26/2016 01:36 PM, Natxo Asenjo wrote:

And in my example, the replica id would be 66, 96, 71 and 97, correct?

no, I don't think so. you searched 2 times the same host "-h
kdc04.unix.iriszorg.nl ".
you need to search on kdc03 to find the current replicaid of
kdc03 and you have to keep it.



yes, you are right :(

 $ ldapsearch -Z -h kdc03.unix.iriszorg.nl
 -D "cn=Directory Manager" -W -b
"o=ipaca"

"(&(objectclass=nstombstone)(nsUniqueId=---))"
| grep "nsds50ruv\|nsDS5ReplicaId"
Enter LDAP Password:
nsDS5ReplicaId: 66
nsds50ruv: {replicageneration} 50c1015c0060
nsds50ruv: {replica 66 ldap://kdc03.unix.iriszorg.nl:389
} 57e23f660042
nsds50ruv: {replica 1095 ldap://kdc04.unix.iriszorg.nl:389
} 57e4d75a044700
nsds50ruv: {replica 96 ldap://kdc01.unix.iriszorg.nl:7389
} 50c1016c006
nsds50ruv: {replica 71 ldap://kdc03.unix.iriszorg.nl:389
} 57e140c70047
nsds50ruv: {replica 97 ldap://kdc02.unix.iriszorg.nl:7389
} 50c101680061000


so I need to keep 66 and 1095, and run the task on 96, 71 and 97,
it would seem.

Thanks for spotting my error.



ok, so I have now run the commands against both ldap hosts (the kdc03 
and the kdc04), and now I have this:
you need to run it only against one host, it will propagate itself to 
the other replicas, if it can - see below.


 # ldapsearch -Z -h kdc04.unix.iriszorg.nl 
 -D "cn=Directory Manager" -W -b 
"o=ipaca" 
"(&(objectclass=nstombstone)(nsUniqueId=---))" 
| grep "nsds50ruv\|nsDS5ReplicaId"

Enter LDAP Password:
nsDS5ReplicaId: 1095
nsds50ruv: {replicageneration} 50c1015c0060
nsds50ruv: {replica 1095 ldap://kdc04.unix.iriszorg.nl:389 
} 57e4d75a044700
nsds50ruv: {replica 66 ldap://kdc03.unix.iriszorg.nl:389 
} 57e23f660042


# ldapsearch -Z -h kdc03.unix.iriszorg.nl 
 -D "cn=Directory Manager" -W -b 
"o=ipaca" 
"(&(objectclass=nstombstone)(nsUniqueId=---))" 
| grep "nsds50ruv\|nsDS5ReplicaId"

Enter LDAP Password:
nsDS5ReplicaId: 66
nsds50ruv: {replicageneration} 50c1015c0060
nsds50ruv: {replica 66 ldap://kdc03.unix.iriszorg.nl:389 
} 57e23f660042
nsds50ruv: {replica 1095 ldap://kdc04.unix.iriszorg.nl:389 
} 57e4d75a044700
nsds50ruv: {replica 96 ldap://kdc01.unix.iriszorg.nl:7389 
} 50c1016c006
nsds50ruv: {replica 71 ldap://kdc03.unix.iriszorg.nl:389 
} 57e140c70047
nsds50ruv: {replica 97 ldap://kdc02.unix.iriszorg.nl:7389 
} 50c101680061000


so the command has not been successful in the kdc03. in the dirsrv 
errors log  I see:


[26/Sep/2016:14:50:54 +0200] NSMMReplicationPlugin - CleanAllRUV Task 
(rid 71): Not all replicas online, retrying in 640 seconds...
this looks like there is still a replication agreement to one of the no 
longer existing servers.


can you search for "... -b "cn=config" 
"objectclass=nsds5replicationagreement"


and remove the ones no longer needed.
[26/Sep/2016:14:51:00 +0200] slapi_ldap_bind - Error: could not send 
startTLS request: error -1 (Can't contact LDAP server) errno 107 
(Transport endpoint is not connected)


but those replicas are gone (decommissioned). So how can I remove them?





--
regards,
Natxo





--
--
Groeten,
natxo




--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander

-- 
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

Re: [Freeipa-users] replicas removed, but incorrectly

2016-09-26 Thread Natxo Asenjo 
On Mon, Sep 26, 2016 at 1:54 PM, Natxo Asenjo 
wrote:

>
>
>
> On Mon, Sep 26, 2016 at 1:50 PM, Ludwig Krispenz 
> wrote:
>
>>
>> On 09/26/2016 01:36 PM, Natxo Asenjo wrote:
>>
>> And in my example, the replica id would be 66, 96, 71 and 97, correct?
>>
>> no, I don't think so. you searched 2 times the same host "-h
>> kdc04.unix.iriszorg.nl".
>> you need to search on kdc03 to find the current replicaid of kdc03 and
>> you have to keep it.
>>
>
>
> yes, you are right :(
>
>  $ ldapsearch -Z -h kdc03.unix.iriszorg.nl -D "cn=Directory Manager" -W
> -b "o=ipaca" 
> "(&(objectclass=nstombstone)(nsUniqueId=---))"
> | grep "nsds50ruv\|nsDS5ReplicaId"
> Enter LDAP Password:
> nsDS5ReplicaId: 66
> nsds50ruv: {replicageneration} 50c1015c0060
> nsds50ruv: {replica 66 ldap://kdc03.unix.iriszorg.nl:389}
> 57e23f660042
> nsds50ruv: {replica 1095 ldap://kdc04.unix.iriszorg.nl:389}
> 57e4d75a044700
> nsds50ruv: {replica 96 ldap://kdc01.unix.iriszorg.nl:7389}
> 50c1016c006
> nsds50ruv: {replica 71 ldap://kdc03.unix.iriszorg.nl:389}
> 57e140c70047
> nsds50ruv: {replica 97 ldap://kdc02.unix.iriszorg.nl:7389}
> 50c101680061000
>
>
> so I need to keep 66 and 1095, and run the task on 96, 71 and 97, it would
> seem.
>
> Thanks for spotting my error.
>


ok, so I have now run the commands against both ldap hosts (the kdc03 and
the kdc04), and now I have this:

 # ldapsearch -Z -h kdc04.unix.iriszorg.nl -D "cn=Directory Manager" -W -b
"o=ipaca"
"(&(objectclass=nstombstone)(nsUniqueId=---))"
| grep "nsds50ruv\|nsDS5ReplicaId"
Enter LDAP Password:
nsDS5ReplicaId: 1095
nsds50ruv: {replicageneration} 50c1015c0060
nsds50ruv: {replica 1095 ldap://kdc04.unix.iriszorg.nl:389}
57e4d75a044700
nsds50ruv: {replica 66 ldap://kdc03.unix.iriszorg.nl:389}
57e23f660042

# ldapsearch -Z -h kdc03.unix.iriszorg.nl -D "cn=Directory Manager" -W -b
"o=ipaca"
"(&(objectclass=nstombstone)(nsUniqueId=---))"
| grep "nsds50ruv\|nsDS5ReplicaId"
Enter LDAP Password:
nsDS5ReplicaId: 66
nsds50ruv: {replicageneration} 50c1015c0060
nsds50ruv: {replica 66 ldap://kdc03.unix.iriszorg.nl:389}
57e23f660042
nsds50ruv: {replica 1095 ldap://kdc04.unix.iriszorg.nl:389}
57e4d75a044700
nsds50ruv: {replica 96 ldap://kdc01.unix.iriszorg.nl:7389}
50c1016c006
nsds50ruv: {replica 71 ldap://kdc03.unix.iriszorg.nl:389}
57e140c70047
nsds50ruv: {replica 97 ldap://kdc02.unix.iriszorg.nl:7389}
50c101680061000

so the command has not been successful in the kdc03. in the dirsrv errors
log  I see:

[26/Sep/2016:14:50:54 +0200] NSMMReplicationPlugin - CleanAllRUV Task (rid
71): Not all replicas online, retrying in 640 seconds...
[26/Sep/2016:14:51:00 +0200] slapi_ldap_bind - Error: could not send
startTLS request: error -1 (Can't contact LDAP server) errno 107 (Transport
endpoint is not connected)

but those replicas are gone (decommissioned). So how can I remove them?


-- 
regards,
Natxo





-- 
--
Groeten,
natxo
-- 
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

Re: [Freeipa-users] replicas removed, but incorrectly

2016-09-26 Thread Natxo Asenjo 
On Mon, Sep 26, 2016 at 1:50 PM, Ludwig Krispenz 
wrote:

>
> On 09/26/2016 01:36 PM, Natxo Asenjo wrote:
>
> hi,
>
> I recently upgraded a centos 6.8 realm to centos 7.2 and it almost went
> correctly.
>
> Now I see some errors in /var/log/dirsrv/slapd-INSTANCENAME/errors
>
> 26/Sep/2016:13:20:15 +0200] attrlist_replace - attr_replace
> (nsslapd-referral, ldap://kdc03.unix.iriszorg.nl:389/o%3Dipaca) failed
>
> and according to http://www.freeipa.org/page/Troubleshooting#Replication_
> issues this points to a ruv problem.
>
> So let's enumerate.
>
> We had kdc01 replicating to kdc02 (both 6.8).
>
> Then I created a replica from kdc01 to kdc03 (running 7.2).
>
> And from kdc03 to kdc04 (both 7.2).
>
> kdc01 and kdc02 are decommissioned, but kdc02 still shows in both kdc03
> and kdc04:
>
> $ ipa-replica-manage list
> kdc02.unix.iriszorg.nl: master
> kdc03.unix.iriszorg.nl: master
> kdc04.unix.iriszorg.nl: master
>
> and in
>
> $ ipa-csreplica-manage list
> Directory Manager password:
> kdc02.unix.iriszorg.nl: master
> kdc03.unix.iriszorg.nl: master
> kdc04.unix.iriszorg.nl: master
>
>
> >From kdc03:
> $ ldapsearch -Z -h kdc04.unix.iriszorg.nl -D "cn=Directory Manager" -W -b
> "o=ipaca" 
> "(&(objectclass=nstombstone)(nsUniqueId=---))"
> | grep "nsds50ruv\|nsDS5ReplicaId"
> Enter LDAP Password:
> nsDS5ReplicaId: 1095
> nsds50ruv: {replicageneration} 50c1015c0060
> nsds50ruv: {replica 1095 ldap://kdc04.unix.iriszorg.nl:389}
> 57e4d75a044700
> nsds50ruv: {replica 66 ldap://kdc03.unix.iriszorg.nl:389}
> 57e23f660042
> nsds50ruv: {replica 96 ldap://kdc01.unix.iriszorg.nl:7389}
> 50c1016c006
> nsds50ruv: {replica 71 ldap://kdc03.unix.iriszorg.nl:389}
> 57e140c70047
> nsds50ruv: {replica 97 ldap://kdc02.unix.iriszorg.nl:7389}
> 50c101680061000
>
> and from kdc04:
>
> # ldapsearch -Z -h kdc04.unix.iriszorg.nl -D "cn=Directory Manager" -W -b
> "o=ipaca" 
> "(&(objectclass=nstombstone)(nsUniqueId=---))"
> | grep "nsds50ruv\|nsDS5ReplicaId"
> Enter LDAP Password:
> nsDS5ReplicaId: 1095
> nsds50ruv: {replicageneration} 50c1015c0060
> nsds50ruv: {replica 1095 ldap://kdc04.unix.iriszorg.nl:389}
> 57e4d75a044700
> nsds50ruv: {replica 66 ldap://kdc03.unix.iriszorg.nl:389}
> 57e23f660042
> nsds50ruv: {replica 96 ldap://kdc01.unix.iriszorg.nl:7389}
> 50c1016c006
> nsds50ruv: {replica 71 ldap://kdc03.unix.iriszorg.nl:389}
> 57e140c70047
> nsds50ruv: {replica 97 ldap://kdc02.unix.iriszorg.nl:7389}
> 50c101680061000
>
>
> So now I have to run a clen ruv task like this (as seen in
> https://www.redhat.com/archives/freeipa-users/2016-May/msg00043.html):
>
> # ldapmodify -ZZ -D "cn=directory manager" -W -a
> dn: cn=clean 13, cn=cleanallruv, cn=tasks, cn=config
> objectclass: extensibleObject
> replica-base-dn: o=ipaca
> replica-id: 13
> cn: clean 13
>
>
> And in my example, the replica id would be 66, 96, 71 and 97, correct?
>
> no, I don't think so. you searched 2 times the same host "-h
> kdc04.unix.iriszorg.nl".
> you need to search on kdc03 to find the current replicaid of kdc03 and you
> have to keep it.
>


yes, you are right :(

 $ ldapsearch -Z -h kdc03.unix.iriszorg.nl -D "cn=Directory Manager" -W -b
"o=ipaca"
"(&(objectclass=nstombstone)(nsUniqueId=---))"
| grep "nsds50ruv\|nsDS5ReplicaId"
Enter LDAP Password:
nsDS5ReplicaId: 66
nsds50ruv: {replicageneration} 50c1015c0060
nsds50ruv: {replica 66 ldap://kdc03.unix.iriszorg.nl:389}
57e23f660042
nsds50ruv: {replica 1095 ldap://kdc04.unix.iriszorg.nl:389}
57e4d75a044700
nsds50ruv: {replica 96 ldap://kdc01.unix.iriszorg.nl:7389}
50c1016c006
nsds50ruv: {replica 71 ldap://kdc03.unix.iriszorg.nl:389}
57e140c70047
nsds50ruv: {replica 97 ldap://kdc02.unix.iriszorg.nl:7389}
50c101680061000


so I need to keep 66 and 1095, and run the task on 96, 71 and 97, it would
seem.

Thanks for spotting my error.

-- 
regards,
natxo
-- 
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

Re: [Freeipa-users] replicas removed, but incorrectly

2016-09-26 Thread Ludwig Krispenz 


On 09/26/2016 01:36 PM, Natxo Asenjo wrote:

hi,

I recently upgraded a centos 6.8 realm to centos 7.2 and it almost 
went correctly.


Now I see some errors in /var/log/dirsrv/slapd-INSTANCENAME/errors

26/Sep/2016:13:20:15 +0200] attrlist_replace - attr_replace 
(nsslapd-referral, ldap://kdc03.unix.iriszorg.nl:389/o%3Dipaca 
) failed


and according to 
http://www.freeipa.org/page/Troubleshooting#Replication_issues this 
points to a ruv problem.


So let's enumerate.

We had kdc01 replicating to kdc02 (both 6.8).

Then I created a replica from kdc01 to kdc03 (running 7.2).

And from kdc03 to kdc04 (both 7.2).

kdc01 and kdc02 are decommissioned, but kdc02 still shows in both 
kdc03 and kdc04:


$ ipa-replica-manage list
kdc02.unix.iriszorg.nl : master
kdc03.unix.iriszorg.nl : master
kdc04.unix.iriszorg.nl : master

and in

$ ipa-csreplica-manage list
Directory Manager password:
kdc02.unix.iriszorg.nl : master
kdc03.unix.iriszorg.nl : master
kdc04.unix.iriszorg.nl : master


>From kdc03:
$ ldapsearch -Z -h kdc04.unix.iriszorg.nl 
 -D "cn=Directory Manager" -W -b 
"o=ipaca" 
"(&(objectclass=nstombstone)(nsUniqueId=---))" 
| grep "nsds50ruv\|nsDS5ReplicaId"

Enter LDAP Password:
nsDS5ReplicaId: 1095
nsds50ruv: {replicageneration} 50c1015c0060
nsds50ruv: {replica 1095 ldap://kdc04.unix.iriszorg.nl:389 
} 57e4d75a044700
nsds50ruv: {replica 66 ldap://kdc03.unix.iriszorg.nl:389 
} 57e23f660042
nsds50ruv: {replica 96 ldap://kdc01.unix.iriszorg.nl:7389 
} 50c1016c006
nsds50ruv: {replica 71 ldap://kdc03.unix.iriszorg.nl:389 
} 57e140c70047
nsds50ruv: {replica 97 ldap://kdc02.unix.iriszorg.nl:7389 
} 50c101680061000


and from kdc04:

# ldapsearch -Z -h kdc04.unix.iriszorg.nl 
 -D "cn=Directory Manager" -W -b 
"o=ipaca" 
"(&(objectclass=nstombstone)(nsUniqueId=---))" 
| grep "nsds50ruv\|nsDS5ReplicaId"

Enter LDAP Password:
nsDS5ReplicaId: 1095
nsds50ruv: {replicageneration} 50c1015c0060
nsds50ruv: {replica 1095 ldap://kdc04.unix.iriszorg.nl:389 
} 57e4d75a044700
nsds50ruv: {replica 66 ldap://kdc03.unix.iriszorg.nl:389 
} 57e23f660042
nsds50ruv: {replica 96 ldap://kdc01.unix.iriszorg.nl:7389 
} 50c1016c006
nsds50ruv: {replica 71 ldap://kdc03.unix.iriszorg.nl:389 
} 57e140c70047
nsds50ruv: {replica 97 ldap://kdc02.unix.iriszorg.nl:7389 
} 50c101680061000



So now I have to run a clen ruv task like this (as seen in 
https://www.redhat.com/archives/freeipa-users/2016-May/msg00043.html):


# ldapmodify -ZZ -D "cn=directory manager" -W -a
dn: cn=clean 13, cn=cleanallruv, cn=tasks, cn=config
objectclass: extensibleObject
replica-base-dn: o=ipaca
replica-id: 13
cn: clean 13

And in my example, the replica id would be 66, 96, 71 and 97, correct?
no, I don't think so. you searched 2 times the same host "-h 
kdc04.unix.iriszorg.nl ".
you need to search on kdc03 to find the current replicaid of kdc03 and 
you have to keep it.

Thanks for confirming this, never done it before.
--
Groeten,
natxo




--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander

-- 
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project