Re: [Freeipa-users] IPA replica issue
On 02/06/2017 05:14 PM, Giorgio Biacchi wrote: On 02/06/2017 04:54 PM, Rob Crittenden wrote: Giorgio Biacchi wrote: Hi list, I have this message in the logs: Feb 6 16:43:10 dc01 ns-slapd: [06/Feb/2017:16:43:10.157801305 +0100] NSMMReplicationPlugin - agmt="cn=masterAgreement1-dc02.myorg.local-pki-tomcat" (dc02:389): Data required to update replica has been purged from the changelog. The replica must be reinitialized. But ipa-replica-manage re-initialize --from dc02.myorg.local does not fix the problem. Even moving away the changelog directory didn't help.. I'm running ipa-server-4.4.0-14.el7.centos.4.x86_64 and 389-ds-base-1.3.5.10-15.el7_3.x86_64, and setup is: #ipa-replica-manage list Directory Manager password: dc01.myorg.local: master dc02.myorg.local: master Can someone please tell me which is the correct sequence of actions to fix this issue? The error appears to be the CA replicated data (ref to tomcat in the agreement) so you need to use ipa-csreplica-manage instead of ipa-replica-manage. rob Hi Rob, even ipa-csreplica-manage re-initialize --from dc02.myorg.local seems not to solve the issue, here's the logs after the command you suggested: Feb 6 17:12:06 dc01 ns-slapd: [06/Feb/2017:17:12:06.432485541 +0100] NSMMReplicationPlugin - changelog program - agmt="cn=meTodc02.myorg.local" (idc02:389): CSN 58989367000c0004 not found, we aren't as up to date, or we purged Feb 6 17:12:06 dc01 ns-slapd: [06/Feb/2017:17:12:06.436444629 +0100] NSMMReplicationPlugin - agmt="cn=meTodc02.myorg.local" (dc02:389): Data required to update replica has been purged from the changelog. The replica must be reinitialized. Thanks for your kind attention Hello again, after a couple of re-initialization (ipa-csreplica-manage and ipa-replica-manage) and after systemctl restart ipa now the previuos error is gone and the replica is working in both directions. Now I have a new error: Feb 6 18:02:12 dc01 [sssd[ldap_child[10109]]]: Failed to initialize credentials using keytab [MEMORY:/etc/krb5.keytab]: Decrypt integrity check failed. Unable to create GSSAPI-encrypted LDAP connection. Feb 6 18:02:12 dc01 [sssd[ldap_child[10109]]]: Decrypt integrity check failed There's a way to fix this?? Thanks -- gb PGP Key: http://pgp.mit.edu/ Primary key fingerprint: C510 0765 943E EBED A4F2 69D3 16CC DC90 B9CB 0F34 -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] IPA replica issue
On 02/06/2017 04:54 PM, Rob Crittenden wrote: Giorgio Biacchi wrote: Hi list, I have this message in the logs: Feb 6 16:43:10 dc01 ns-slapd: [06/Feb/2017:16:43:10.157801305 +0100] NSMMReplicationPlugin - agmt="cn=masterAgreement1-dc02.myorg.local-pki-tomcat" (dc02:389): Data required to update replica has been purged from the changelog. The replica must be reinitialized. But ipa-replica-manage re-initialize --from dc02.myorg.local does not fix the problem. Even moving away the changelog directory didn't help.. I'm running ipa-server-4.4.0-14.el7.centos.4.x86_64 and 389-ds-base-1.3.5.10-15.el7_3.x86_64, and setup is: #ipa-replica-manage list Directory Manager password: dc01.myorg.local: master dc02.myorg.local: master Can someone please tell me which is the correct sequence of actions to fix this issue? The error appears to be the CA replicated data (ref to tomcat in the agreement) so you need to use ipa-csreplica-manage instead of ipa-replica-manage. rob Hi Rob, even ipa-csreplica-manage re-initialize --from dc02.myorg.local seems not to solve the issue, here's the logs after the command you suggested: Feb 6 17:12:06 dc01 ns-slapd: [06/Feb/2017:17:12:06.432485541 +0100] NSMMReplicationPlugin - changelog program - agmt="cn=meTodc02.myorg.local" (idc02:389): CSN 58989367000c0004 not found, we aren't as up to date, or we purged Feb 6 17:12:06 dc01 ns-slapd: [06/Feb/2017:17:12:06.436444629 +0100] NSMMReplicationPlugin - agmt="cn=meTodc02.myorg.local" (dc02:389): Data required to update replica has been purged from the changelog. The replica must be reinitialized. Thanks for your kind attention -- gb PGP Key: http://pgp.mit.edu/ Primary key fingerprint: C510 0765 943E EBED A4F2 69D3 16CC DC90 B9CB 0F34 -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] IPA replica issue
Giorgio Biacchi wrote: > Hi list, > I have this message in the logs: > > Feb 6 16:43:10 dc01 ns-slapd: [06/Feb/2017:16:43:10.157801305 +0100] > NSMMReplicationPlugin - > agmt="cn=masterAgreement1-dc02.myorg.local-pki-tomcat" (dc02:389): Data > required to update replica has been purged from the changelog. The > replica must be reinitialized. > > But ipa-replica-manage re-initialize --from dc02.myorg.local does not > fix the problem. Even moving away the changelog directory didn't help.. > > I'm running ipa-server-4.4.0-14.el7.centos.4.x86_64 and > 389-ds-base-1.3.5.10-15.el7_3.x86_64, and setup is: > > #ipa-replica-manage list > Directory Manager password: > > dc01.myorg.local: master > dc02.myorg.local: master > > Can someone please tell me which is the correct sequence of actions to > fix this issue? The error appears to be the CA replicated data (ref to tomcat in the agreement) so you need to use ipa-csreplica-manage instead of ipa-replica-manage. rob -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
[Freeipa-users] IPA Replica Issue
I have been having replication issues since the update to RHEL6.4 and 389-ds-base-1.2.11.15-12. It is entirely possible that we have more than just 1 problem. Frequently we seeing errors in our replication monitoring indicating: -1 Incremental update has failed and requires administrator actionLDAP error: Can't contact LDAP server This problem cannot be solved via ipa-replication-managment force-sync and it does not get permanently solved with a re-initializeation or a dirsrv restart either (the problem eventually comes back or appears on a different server) Have any of you also seen this error when you could verify that the servers can communicate over ldap? When checking with Rich today in IRC, we turned on debugging for replication and did not see a smoking gun. We -did- see log messages showing things like: (auth1:389): CSN 51ad2c5500090066 not found, we aren't as up to date, or we purged When looking for this change, it was determined that the originating IPA server who was responsible for the change show that this was a modification by the MemberOf plugin associating a host with a hostgroup or vice versa. This change was -not- found on the IPA server who is reporting the replication troubles. IPA deliberately excludes memberof changes during incremental updates for performance reasons. This is because each server does replicate the 'member' info, where by the local MemberOf plugin will fire off and perform its respective fixups accordingly. Rich asked me to bring this issue up to the attention of the mailing list so that we could continue to track the root cause of the issue(s) and hopefully come to a conclusion about how to fix them. Keeping your head in the cloud ~ Jr Aquino | Sr. Information Security Specialist GXPN | GIAC Exploit Researcher and Advanced Penetration Tester GCIH | GIAC Certified Incident Handler GWAPT | GIAC WebApp Penetration Tester Citrix Online | 7408 Hollister Avenue | Goleta, CA 93117x-apple-data-detectors://0/0 T: +1 805.690.3478tel:+1%C2%A0805.690.3478 C: +1 805.717.0365tel:+1%20805.717.0365 jr.aqu...@citrix.commailto:jr.aqu...@citrixonline.com http://www.citrixonline.comhttp://www.citrixonline.com/ ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users
Re: [Freeipa-users] IPA Replica Issue
On Jun 5, 2013, at 5:26 PM, Rich Megginson wrote: On 06/05/2013 05:49 PM, JR Aquino wrote: I have been having replication issues since the update to RHEL6.4 and 389-ds-base-1.2.11.15-12. It is entirely possible that we have more than just 1 problem. Frequently we seeing errors in our replication monitoring indicating: -1 Incremental update has failed and requires administrator actionLDAP error: Can't contact LDAP server This problem cannot be solved via ipa-replication-managment force-sync and it does not get permanently solved with a re-initializeation or a dirsrv restart either (the problem eventually comes back or appears on a different server) Have any of you also seen this error when you could verify that the servers can communicate over ldap? When checking with Rich today in IRC, we turned on debugging for replication and did not see a smoking gun. We -did- see log messages showing things like: (auth1:389): CSN 51ad2c5500090066 not found, we aren't as up to date, or we purged On replicaID 0x66 - I think dbscan -f /var/lib/dirsrv/slapd-INST/cldb/xx.db4 will tell you what are the purge and max CSNs, somewhere near the beginning - what are they? I've looked up and down the dbscan output and there is no sign of the word 'purge' or 'max' Also, what is the database RUV on 0x66? that is, do ldapsearch -xLLL -h 0x66hostname -D cn=directory manager -w password -b dc=expertcity,dc=com '((objectclass=nsTombstone)(nsuniqueid=---))' I've sent you a private email from for the above output When looking for this change, it was determined that the originating IPA server who was responsible for the change show that this was a modification by the MemberOf plugin associating a host with a hostgroup or vice versa. This change was -not- found on the IPA server who is reporting the replication troubles. IPA deliberately excludes memberof changes during incremental updates for performance reasons. This is because each server does replicate the 'member' info, where by the local MemberOf plugin will fire off and perform its respective fixups accordingly. Rich asked me to bring this issue up to the attention of the mailing list so that we could continue to track the root cause of the issue(s) and hopefully come to a conclusion about how to fix them. Keeping your head in the cloud ~ Jr Aquino | Sr. Information Security Specialist GXPN | GIAC Exploit Researcher and Advanced Penetration Tester GCIH | GIAC Certified Incident Handler GWAPT | GIAC WebApp Penetration Tester Citrix Online | 7408 Hollister Avenue | Goleta, CA 93117x-apple-data-detectors://0/0 T: +1 805.690.3478tel:+1%C2%A0805.690.3478 C: +1 805.717.0365tel:+1%20805.717.0365 jr.aqu...@citrix.commailto:jr.aqu...@citrixonline.com http://www.citrixonline.comhttp://www.citrixonline.com/ ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users
Re: [Freeipa-users] IPA Replica Issue
On 06/05/2013 07:20 PM, JR Aquino wrote: On Jun 5, 2013, at 5:26 PM, Rich Megginson wrote: On 06/05/2013 05:49 PM, JR Aquino wrote: I have been having replication issues since the update to RHEL6.4 and 389-ds-base-1.2.11.15-12. It is entirely possible that we have more than just 1 problem. Frequently we seeing errors in our replication monitoring indicating: -1 Incremental update has failed and requires administrator actionLDAP error: Can't contact LDAP server This problem cannot be solved via ipa-replication-managment force-sync and it does not get permanently solved with a re-initializeation or a dirsrv restart either (the problem eventually comes back or appears on a different server) Have any of you also seen this error when you could verify that the servers can communicate over ldap? When checking with Rich today in IRC, we turned on debugging for replication and did not see a smoking gun. We -did- see log messages showing things like: (auth1:389): CSN 51ad2c5500090066 not found, we aren't as up to date, or we purged On replicaID 0x66 - I think dbscan -f /var/lib/dirsrv/slapd-INST/cldb/xx.db4 will tell you what are the purge and max CSNs, somewhere near the beginning - what are they? I've looked up and down the dbscan output and there is no sign of the word 'purge' or 'max' ok - try this dbscan -k 00de -f /var/lib/dirsrv/slapd-INST/cldb/xx.db4 and dbscan -k 014d -f /var/lib/dirsrv/slapd-INST/cldb/xx.db4 If that gives you nothing, then just tell me what the first and last csns are. Also, what is the database RUV on 0x66? that is, do ldapsearch -xLLL -h 0x66hostname -D cn=directory manager -w password -b dc=expertcity,dc=com '((objectclass=nsTombstone)(nsuniqueid=---))' I've sent you a private email from for the above output When looking for this change, it was determined that the originating IPA server who was responsible for the change show that this was a modification by the MemberOf plugin associating a host with a hostgroup or vice versa. This change was -not- found on the IPA server who is reporting the replication troubles. IPA deliberately excludes memberof changes during incremental updates for performance reasons. This is because each server does replicate the 'member' info, where by the local MemberOf plugin will fire off and perform its respective fixups accordingly. Rich asked me to bring this issue up to the attention of the mailing list so that we could continue to track the root cause of the issue(s) and hopefully come to a conclusion about how to fix them. Keeping your head in the cloud ~ Jr Aquino | Sr. Information Security Specialist GXPN | GIAC Exploit Researcher and Advanced Penetration Tester GCIH | GIAC Certified Incident Handler GWAPT | GIAC WebApp Penetration Tester Citrix Online | 7408 Hollister Avenue | Goleta, CA 93117x-apple-data-detectors://0/0 T: +1 805.690.3478tel:+1%C2%A0805.690.3478 C: +1 805.717.0365tel:+1%20805.717.0365 jr.aqu...@citrix.commailto:jr.aqu...@citrixonline.com http://www.citrixonline.comhttp://www.citrixonline.com/ ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users
