re-initialize was not sufficient.
We needed to trash ipa34 and ipa35 completely and install new replica.
We had some hassle with hosts file and permission denied on the
ipa-replica-install step,
in the end it worked out.
--
*Sándor Juhász*
System Administrator
*ChemAxon* *Ltd*.
Building Hx, GraphiSoft Park, Záhony utca 7, Budapest, Hungary, H-1031
Cell: +36704258964
On Fri, Apr 13, 2018 at 11:56 AM Ludwig Krispenz <lkris...@redhat.com>
wrote:
> Hi,
> when extracting the relevant data, we see:
>
> [root@ipa14 ~]
> dn: cn=replica,cn=o\3Dipaca,cn=mapping tree,cn=config
> nsds5replicaid: 6
> nsds50ruv: {replicageneration} 58987e19000000060000
> nsds50ruv: {replica 6 ldap://ipa14.bpo.cxn:389} 58987e1c000000060000
> 5ad07153000000060000
> nsds50ruv: {replica 16 ldap://ipa15.bpo.cxn:389} 5a0c4f48000000100000
> 5a0da16d000200100000
> nsds50ruv: {replica 12 ldap://ipa34.bph.cxn:389} 59d74be60000000c0000
> 59d74c4e0004000c0000
> nsds50ruv: {replica 8 ldap://ipa35.bph.cxn:389} 5898a4e0000000080000
> 589adeca000000080000
> ~~~~~~~~
> [root@ipa15 ~]
> dn: cn=replica,cn=o\3Dipaca,cn=mapping tree,cn=config
> nsds5replicaid: 16
> nsds50ruv: {replicageneration} 58987e19000000060000
> nsds50ruv: {replica 16 ldap://ipa15.bpo.cxn:389} 5a0c4f48000000100000
> 5a0da16d000200100000
> nsds50ruv: {replica 6 ldap://ipa14.bpo.cxn:389} 58987e1c000000060000
> 5ad07153000000060000
> nsds50ruv: {replica 12 ldap://ipa34.bph.cxn:389} 59d74be60000000c0000
> 59d74c4e0004000c0000
> nsds50ruv: {replica 8 ldap://ipa35.bph.cxn:389} 5898a4e0000000080000
> 589adeca000000080000
> ~~~~~~~~
> [root@ipa34 ~]
> dn: cn=replica,cn=o\3Dipaca,cn=mapping tree,cn=config
> nsds5replicaid: 12
> nsds50ruv: {replicageneration} 58987e19000000060000
> nsds50ruv: {replica 12 ldap://ipa34.bph.cxn:389} 59d74be60000000c0000
> 59d74c4e0004000c0000
> nsds50ruv: {replica 16 ldap://ipa15.bpo.cxn:389}
> nsds50ruv: {replica 6 ldap://ipa14.bpo.cxn:389} 58987e1c000000060000
> 5a0a27d9000000060000
> nsds50ruv: {replica 8 ldap://ipa35.bph.cxn:389} 5898a4e0000000080000
> 589adeca000000080000
> ~~~~~~~~
> [root@ipa35 ~]
> dn: cn=replica,cn=o\3Dipaca,cn=mapping tree,cn=config
> nsds5replicaid: 8
> nsds50ruv: {replicageneration} 58987e19000000060000
> nsds50ruv: {replica 8 ldap://ipa35.bph.cxn:389} 5898a4e0000000080000
> 589adeca000000080000
> nsds50ruv: {replica 16 ldap://ipa15.bpo.cxn:389}
> nsds50ruv: {replica 6 ldap://ipa14.bpo.cxn:389} 58987e1c000000060000
> 5a0a27d9000000060000
> nsds50ruv: {replica 12 ldap://ipa34.bph.cxn:389} 59d74be60000000c0000
> 59d74c4e0004000c0000
>
>
> this indicates that all replicas are in sync for replicaid 8 and 12, but
> for rid 16, ipa 34 and ipa 35 have no data and for rid 6 they have older
> data.
> I cannot say what has happened, but I think you need reinit 34 and 35 from
> either 14 or 15
>
> On 04/13/2018 11:13 AM, Sandor Juhasz wrote:
>
> here are the results:
>
> ~~~~~~~~
>
> [root@ipa14 ~]# ldapsearch -H ldap://ipa14.bpo.cxn -o ldif-wrap=no -D
> "cn=directory manager" -x -W -b cn=config "objectclass=nsds5replica"
> nsds5replicaid nsds50ruv
> Enter LDAP Password:
> # extended LDIF
> #
> # LDAPv3
> # base <cn=config> with scope subtree
> # filter: objectclass=nsds5replica
> # requesting: nsds5replicaid nsds50ruv
> #
>
> # replica, dc\3Dcxn, mapping tree, config
> dn: cn=replica,cn=dc\3Dcxn,cn=mapping tree,cn=config
> nsds5replicaid: 4
> nsds50ruv: {replicageneration} 58987d9e000000040000
> nsds50ruv: {replica 4 ldap://ipa14.bpo.cxn:389} 58987d9e000100040000
> 5ad07160000000040000
> nsds50ruv: {replica 7 ldap://ipa35.bph.cxn:389} 5898a473000000070000
> 5ad06adb000900070000
> nsds50ruv: {replica 11 ldap://ipa34.bph.cxn:389} 59d74b730000000b0000
> 5ad0711c003a000b0000
> nsds50ruv: {replica 15 ldap://ipa15.bpo.cxn:389} 5a0c4ed00000000f0000
> 5ad06e1a0004000f0000
>
> # replica, o\3Dipaca, mapping tree, config
> dn: cn=replica,cn=o\3Dipaca,cn=mapping tree,cn=config
> nsds5replicaid: 6
> nsds50ruv: {replicageneration} 58987e19000000060000
> nsds50ruv: {replica 6 ldap://ipa14.bpo.cxn:389} 58987e1c000000060000
> 5ad07153000000060000
> nsds50ruv: {replica 16 ldap://ipa15.bpo.cxn:389} 5a0c4f48000000100000
> 5a0da16d000200100000
> nsds50ruv: {replica 12 ldap://ipa34.bph.cxn:389} 59d74be60000000c0000
> 59d74c4e0004000c0000
> nsds50ruv: {replica 8 ldap://ipa35.bph.cxn:389} 5898a4e0000000080000
> 589adeca000000080000
>
> # search result
> search: 2
> result: 0 Success
>
> # numResponses: 3
> # numEntries: 2
> [root@ipa14 ~]#
>
> ~~~~~~~~
>
> [root@ipa15 ~]# ldapsearch -H ldap://ipa15 -o ldif-wrap=no -D
> "cn=directory manager" -x -W -b cn=config "objectclass=nsds5replica"
> nsds5replicaid nsds50ruv
> Enter LDAP Password:
> # extended LDIF
> #
> # LDAPv3
> # base <cn=config> with scope subtree
> # filter: objectclass=nsds5replica
> # requesting: nsds5replicaid nsds50ruv
> #
>
> # replica, dc\3Dcxn, mapping tree, config
> dn: cn=replica,cn=dc\3Dcxn,cn=mapping tree,cn=config
> nsds5replicaid: 15
> nsds50ruv: {replicageneration} 58987d9e000000040000
> nsds50ruv: {replica 15 ldap://ipa15.bpo.cxn:389} 5a0c4ed00000000f0000
> 5ad071c20000000f0000
> nsds50ruv: {replica 7 ldap://ipa35.bph.cxn:389} 5898a473000000070000
> 5ad06adb000900070000
> nsds50ruv: {replica 4 ldap://ipa14.bpo.cxn:389} 58987d9e000100040000
> 5ad071af002d00040000
> nsds50ruv: {replica 11 ldap://ipa34.bph.cxn:389} 59d74b730000000b0000
> 5ad071d20021000b0000
>
> # replica, o\3Dipaca, mapping tree, config
> dn: cn=replica,cn=o\3Dipaca,cn=mapping tree,cn=config
> nsds5replicaid: 16
> nsds50ruv: {replicageneration} 58987e19000000060000
> nsds50ruv: {replica 16 ldap://ipa15.bpo.cxn:389} 5a0c4f48000000100000
> 5a0da16d000200100000
> nsds50ruv: {replica 6 ldap://ipa14.bpo.cxn:389} 58987e1c000000060000
> 5ad07153000000060000
> nsds50ruv: {replica 12 ldap://ipa34.bph.cxn:389} 59d74be60000000c0000
> 59d74c4e0004000c0000
> nsds50ruv: {replica 8 ldap://ipa35.bph.cxn:389} 5898a4e0000000080000
> 589adeca000000080000
>
> # search result
> search: 2
> result: 0 Success
>
> # numResponses: 3
> # numEntries: 2
> [root@ipa15 ~]#
>
> ~~~~~~~~
>
> [root@ipa34 ~]# ldapsearch -H ldap://ipa34 -o ldif-wrap=no -D
> "cn=directory manager" -x -W -b cn=config "objectclass=nsds5replica"
> nsds5replicaid nsds50ruv
> Enter LDAP Password:
> # extended LDIF
> #
> # LDAPv3
> # base <cn=config> with scope subtree
> # filter: objectclass=nsds5replica
> # requesting: nsds5replicaid nsds50ruv
> #
>
> # replica, dc\3Dcxn, mapping tree, config
> dn: cn=replica,cn=dc\3Dcxn,cn=mapping tree,cn=config
> nsds5replicaid: 11
> nsds50ruv: {replicageneration} 58987d9e000000040000
> nsds50ruv: {replica 11 ldap://ipa34.bph.cxn:389} 59d74b730000000b0000
> 5ad072120003000b0000
> nsds50ruv: {replica 7 ldap://ipa35.bph.cxn:389} 5898a473000000070000
> 5ad06adb000900070000
> nsds50ruv: {replica 4 ldap://ipa14.bpo.cxn:389} 58987d9e000100040000
> 5ad071af002d00040000
> nsds50ruv: {replica 15 ldap://ipa15.bpo.cxn:389} 5a0c4ed00000000f0000
> 5ad06e1a0004000f0000
>
> # replica, o\3Dipaca, mapping tree, config
> dn: cn=replica,cn=o\3Dipaca,cn=mapping tree,cn=config
> nsds5replicaid: 12
> nsds50ruv: {replicageneration} 58987e19000000060000
> nsds50ruv: {replica 12 ldap://ipa34.bph.cxn:389} 59d74be60000000c0000
> 59d74c4e0004000c0000
> nsds50ruv: {replica 16 ldap://ipa15.bpo.cxn:389}
> nsds50ruv: {replica 6 ldap://ipa14.bpo.cxn:389} 58987e1c000000060000
> 5a0a27d9000000060000
> nsds50ruv: {replica 8 ldap://ipa35.bph.cxn:389} 5898a4e0000000080000
> 589adeca000000080000
>
> # search result
> search: 2
> result: 0 Success
>
> # numResponses: 3
> # numEntries: 2
> [root@ipa34 ~]#
>
> ~~~~~~~~
>
> [root@ipa35 ~]# ldapsearch -H ldap://ipa35 -o ldif-wrap=no -D
> "cn=directory manager" -x -W -b cn=config "objectclass=nsds5replica"
> nsds5replicaid nsds50ruv
> Enter LDAP Password:
> # extended LDIF
> #
> # LDAPv3
> # base <cn=config> with scope subtree
> # filter: objectclass=nsds5replica
> # requesting: nsds5replicaid nsds50ruv
> #
>
> # replica, dc\3Dcxn, mapping tree, config
> dn: cn=replica,cn=dc\3Dcxn,cn=mapping tree,cn=config
> nsds5replicaid: 7
> nsds50ruv: {replicageneration} 58987d9e000000040000
> nsds50ruv: {replica 7 ldap://ipa35.bph.cxn:389} 5898a473000000070000
> 5ad07248001800070000
> nsds50ruv: {replica 4 ldap://ipa14.bpo.cxn:389} 58987d9e000100040000
> 5ad071af002d00040000
> nsds50ruv: {replica 11 ldap://ipa34.bph.cxn:389} 59d74b730000000b0000
> 5ad072490010000b0000
> nsds50ruv: {replica 15 ldap://ipa15.bpo.cxn:389} 5a0c4ed00000000f0000
> 5ad06e1a0004000f0000
>
> # replica, o\3Dipaca, mapping tree, config
> dn: cn=replica,cn=o\3Dipaca,cn=mapping tree,cn=config
> nsds5replicaid: 8
> nsds50ruv: {replicageneration} 58987e19000000060000
> nsds50ruv: {replica 8 ldap://ipa35.bph.cxn:389} 5898a4e0000000080000
> 589adeca000000080000
> nsds50ruv: {replica 16 ldap://ipa15.bpo.cxn:389}
> nsds50ruv: {replica 6 ldap://ipa14.bpo.cxn:389} 58987e1c000000060000
> 5a0a27d9000000060000
> nsds50ruv: {replica 12 ldap://ipa34.bph.cxn:389} 59d74be60000000c0000
> 59d74c4e0004000c0000
>
> # search result
> search: 2
> result: 0 Success
>
> # numResponses: 3
> # numEntries: 2
> [root@ipa35 ~]#
>
> ~~~~~~~~
>
> --
> *Sándor Juhász*
> System Administrator
> *ChemAxon* *Ltd*.
> Building Hx, GraphiSoft Park, Záhony utca 7, Budapest, Hungary, H-1031
> Cell: +36704258964
>
> On Fri, Apr 13, 2018 at 10:51 AM, Ludwig Krispenz via FreeIPA-users <
> freeipa-users@lists.fedorahosted.org> wrote:
>
>>
>> On 04/13/2018 08:25 AM, Sandor Juhasz via FreeIPA-users wrote:
>>
>> Hello,
>>
>> we are using freeipa in a 4way multi master replication setup.
>> Servers ipa14,ipa15 and ipa34,ipa35 on
>> CentOS Linux release 7.3.1611 (Core) with version
>> ipa-server-common-4.4.0-14.el7.centos.7.noarch.
>>
>> We have an issue where one of the servers log a missing CSN. It happens
>> even after
>> ipa replication reinitialized.
>> We are guessing that CSN 5a0a27d9000000060000 only exists on ipa35, but
>> we see it in those files listed on ipa15 and the error is reported there.
>> Please see attached file with logs.
>>
>> the missing csn is from Nov,13,2017 - so it is not unlikely it was
>> trimmed. But in some RUV there seems to be a reference to it, and
>> replication uses to position it in the changelog.
>>
>>
>>
>> How can we fix this?
>>
>> we first should get a full picture of the replicaids and RUVs on all
>> servers, could you do on all servers the following search:
>> ldapsearch .... -o ldif-wrap=no -D "cn=directory manager" .... -b
>> cn=config "objectclass=nsds5replica" nsds5replicaid nsds50ruv
>>
>> That should help in deciding what to do.
>>
>> There is also on option to kick an agreement to ingnore a missing change:
>>
>> do the following change on the failing replication agreement, but it
>> would be better to have the data first:
>>
>> ldapmodify ....
>> dn: <agmt>
>> replace: nsds5ReplicaIgnoreMissingChange
>> nsds5ReplicaIgnoreMissingChange: once
>>
>>
>> --
>> *Sándor Juhász*
>> System Administrator
>> *ChemAxon* *Ltd*.
>> Building Hx, GraphiSoft Park, Záhony utca 7, Budapest, Hungary, H-1031
>> Cell: +36704258964
>>
>>
>> _______________________________________________
>> FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org
>> To unsubscribe send an email to freeipa-users-le...@lists.fedorahosted.org
>>
>>
>> --
>> Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
>> Commercial register: Amtsgericht Muenchen, HRB 153243,
>> Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill,
>> Eric Shander
>>
>>
>> _______________________________________________
>> FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org
>> To unsubscribe send an email to
>> freeipa-users-le...@lists.fedorahosted.org
>>
>>
>
> --
> Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn,
> Commercial register: Amtsgericht Muenchen, HRB 153243,
> Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill,
> Eric Shander
>
>
_______________________________________________
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org
To unsubscribe send an email to freeipa-users-le...@lists.fedorahosted.org
