On 08/23/2016 11:52 AM, Ian Harding wrote:
Ah.  I see.  I mixed those up but I see that those would have to be
consistent.

However, I have been trying to beat some invalid RUV to death for a long
time and I can't seem to kill them.

For example, bellevuenfs has 9 and 16 which are invalid:

[ianh@seattlenfs ~]$ ldapsearch -ZZ -h seattlenfs.bpt.rocks -D
"cn=Directory Manager" -W -b "dc=bpt,dc=rocks"
"(&(objectclass=nstombstone)(nsUniqueId=ffffffff-ffffffff-ffffffff-ffffffff))"
| grep "nsds50ruv\|nsDS5ReplicaId"
Enter LDAP Password:
nsDS5ReplicaId: 7
nsds50ruv: {replicageneration} 55c8f364000000040000
nsds50ruv: {replica 7 ldap://seattlenfs.bpt.rocks:389}
568ac3cc000000070000 57
nsds50ruv: {replica 20 ldap://freeipa-sea.bpt.rocks:389}
57b10377000200140000
nsds50ruv: {replica 18 ldap://bpt-nyc1-nfs.bpt.rocks:389}
57a47801000100120000
nsds50ruv: {replica 15 ldap://fremontnis.bpt.rocks:389}
57a403860000000f0000 5
nsds50ruv: {replica 14 ldap://freeipa-dal.bpt.rocks:389}
57a2dccd0000000e0000
nsds50ruv: {replica 17 ldap://edinburghnfs.bpt.rocks:389}
57a422f9000000110000
nsds50ruv: {replica 19 ldap://bellevuenfs.bpt.rocks:389}
57a4f20d000600130000
nsds50ruv: {replica 16 ldap://bellevuenfs.bpt.rocks:389}
57a41706000000100000
nsds50ruv: {replica 9 ldap://bellevuenfs.bpt.rocks:389}
570484ee000000090000 5


So I try to kill them like so:
[ianh@seattlenfs ~]$ ipa-replica-manage clean-ruv 9 --force --cleanup
ipa: WARNING: session memcached servers not running
Clean the Replication Update Vector for bellevuenfs.bpt.rocks:389

Cleaning the wrong replica ID will cause that server to no
longer replicate so it may miss updates while the process
is running. It would need to be re-initialized to maintain
consistency. Be very careful.
Background task created to clean replication data. This may take a while.
This may be safely interrupted with Ctrl+C
^C[ianh@seattlenfs ~]$ ipa-replica-manage clean-ruv 16 --force --cleanup
ipa: WARNING: session memcached servers not running
Clean the Replication Update Vector for bellevuenfs.bpt.rocks:389

Cleaning the wrong replica ID will cause that server to no
longer replicate so it may miss updates while the process
is running. It would need to be re-initialized to maintain
consistency. Be very careful.
Background task created to clean replication data. This may take a while.
This may be safely interrupted with Ctrl+C
^C[ianh@seattlenfs ~]$ ipa-replica-manage list-clean-ruv
ipa: WARNING: session memcached servers not running
CLEANALLRUV tasks
RID 16: Waiting to process all the updates from the deleted replica...
RID 9: Waiting to process all the updates from the deleted replica...

No abort CLEANALLRUV tasks running
[ianh@seattlenfs ~]$ ipa-replica-manage list-clean-ruv
ipa: WARNING: session memcached servers not running
CLEANALLRUV tasks
RID 16: Waiting to process all the updates from the deleted replica...
RID 9: Waiting to process all the updates from the deleted replica...

and it never finishes.

seattlenfs is the first master, that's the only place I should have to
run this command, right?
right, you need to run it only on one master, but this ease of use can become the problem. The cleanallruv task is propagated to all servers in the topology and it does this based on the replication agreements it finds. A frequent cause of failure is that replication agreements still exist pointing to no longer existing servers. It is a bit tedious, but could you run the following search on ALL
of your current replicas (as directory manager):

ldapsearch ...... -b "cn=config" "objectclass=nsds5replicationagreement" nsds5replicahost

if you find any agreement where nsds5replicahost is a host no longer existing or working, delete these agreements.

I'm about to burn everything down and ipa-server-install --uninstall but
I've done that before a couple times and that seems to be what got me
into this mess...

Thank you for your help.




On 08/23/2016 01:37 AM, Ludwig Krispenz wrote:
looks like you are searching the nstombstone below "o=ipaca", but you
are cleaning ruvs in "dc=bpt,dc=rocks",

your attrlist_replace error refers to the bpt,rocks backend, so you
should search the tombstone entry ther, then determine which replicaIDs
to remove.

Ludwig

On 08/23/2016 09:20 AM, Ian Harding wrote:
I've followed the procedure in this thread:

https://www.redhat.com/archives/freeipa-users/2016-May/msg00043.html

and found my list of RUV that don't have an existing replica id.

I've tried to remove them like so:

[root@seattlenfs ianh]# ldapmodify -D "cn=directory manager" -W -a
Enter LDAP Password:
dn: cn=clean 97, cn=cleanallruv, cn=tasks, cn=config
objectclass: top
objectclass: extensibleObject
replica-base-dn: dc=bpt,dc=rocks
replica-id: 97
replica-force-cleaning: yes
cn: clean 97

adding new entry "cn=clean 97, cn=cleanallruv, cn=tasks, cn=config"

[root@seattlenfs ianh]# ipa-replica-manage list-clean-ruv
CLEANALLRUV tasks
RID 9: Waiting to process all the updates from the deleted replica...
RID 96: Successfully cleaned rid(96).
RID 97: Successfully cleaned rid(97).

No abort CLEANALLRUV tasks running


and yet, they are still there...

[root@seattlenfs ianh]# ldapsearch -ZZ -h seattlenfs.bpt.rocks -D
"cn=Directory Manager" -W -b "o=ipaca"
"(&(objectclass=nstombstone)(nsUniqueId=ffffffff-ffffffff-ffffffff-ffffffff))"

| grep "nsds50ruv\|nsDS5ReplicaId"
Enter LDAP Password:
nsDS5ReplicaId: 81
nsds50ruv: {replicageneration} 55c8f3ae000000600000
nsds50ruv: {replica 81 ldap://seattlenfs.bpt.rocks:389}
568ac431000000510000 5
nsds50ruv: {replica 1065 ldap://freeipa-sea.bpt.rocks:389}
57b103d400000429000
nsds50ruv: {replica 1070 ldap://bellevuenfs.bpt.rocks:389}
57a4f2700000042e000
nsds50ruv: {replica 1075 ldap://bpt-nyc1-nfs.bpt.rocks:389}
57a478650000043300
nsds50ruv: {replica 1080 ldap://bellevuenfs.bpt.rocks:389}
57a4176700000438000
nsds50ruv: {replica 1085 ldap://fremontnis.bpt.rocks:389}
57a403e60000043d0000
nsds50ruv: {replica 1090 ldap://freeipa-dal.bpt.rocks:389}
57a2dd3500000442000
nsds50ruv: {replica 1095 ldap://freeipa-sea.bpt.rocks:389}
579a963c00000447000
nsds50ruv: {replica 96 ldap://freeipa-sea.bpt.rocks:389}
55c8f3bd000000600000
nsds50ruv: {replica 86 ldap://fremontnis.bpt.rocks:389}
5685b24e000000560000 5
nsds50ruv: {replica 91 ldap://seattlenis.bpt.rocks:389}
567ad6180001005b0000 5
nsds50ruv: {replica 97 ldap://freeipa-dal.bpt.rocks:389}
55c8f3ce000000610000
nsds50ruv: {replica 76 ldap://bellevuenis.bpt.rocks:389}
56f385eb0007004c0000
nsds50ruv: {replica 71 ldap://bellevuenfs.bpt.rocks:389}
57048560000900470000
nsds50ruv: {replica 66 ldap://bpt-nyc1-nfs.bpt.rocks:389}
5733e594000a00420000
nsds50ruv: {replica 61 ldap://edinburghnfs.bpt.rocks:389}
574421250000003d0000
nsds50ruv: {replica 1195 ldap://edinburghnfs.bpt.rocks:389}
57a42390000004ab00

What have I done wrong?

The problem I am trying to solve is that seattlenfs.bpt.rocks sends
updates to all its children, but their changes don't come back because
of these errors:

[23/Aug/2016:00:02:16 -0700] attrlist_replace - attr_replace
(nsslapd-referral,
ldap://seattlenfs.bpt.rocks:389/dc%3Dbpt%2Cdc%3Drocks) failed.

in effect, the replication agreements are one-way.

Any ideas?

- Ian


--
Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric 
Shander

--
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

Reply via email to