Re: [Freeipa-users] CSN not found
On 03/11/16 19:58, Mark Reynolds wrote: dbscan -f /var/lib/dirsrv/slapd-INSTANCE/db/changelogdb >results of above scan do not look like that CSN form reported in >dirsrv's error log, it is: >.. >=116156 >=116157 >=116158 >.. That doesn't look quite right, Just to confirm you should be doing something like dbscan -f /var/lib/dirsrv/slapd-master_1/db/changelogdb/fe665489-a13011e6-acbab8c1-43b12a38_581a3c410001.db | grep 581b120f00050004 I don't see any xx.db in /var/lib/dirsrv/slapd-master_1/db/changelogdb but there are these: 16c9da9e-a54611e6-80ab82b9-81e5c5a8_574596220060.db 16c9da9e-a54611e6-80ab82b9-81e5c5a8.sema DBVERSION e71ad28c-a54511e6-80ab82b9-81e5c5a8_574595c80004.db e71ad28c-a54511e6-80ab82b9-81e5c5a8.sema in /var/lib/dirsrv/slapd-master_1/cldb and if I scant those: cldb]$ for _F in .db; do dbscan -f $_F | grep 57480d6d0025; done there is nothing (on the replica that complains but also nothing on all members) cldb]$ ll ../db/changelog/ total 2260 -rw---. 1 dirsrv dirsrv 16384 Nov 8 00:02 aci.db -rw---. 1 dirsrv dirsrv 40960 Nov 8 15:52 ancestorid.db -rw---. 1 dirsrv dirsrv 40960 Nov 8 15:52 changenumber.db -rw---. 1 dirsrv dirsrv 16384 Nov 8 00:02 cn.db -rw---. 1 dirsrv dirsrv 51 Nov 8 00:02 DBVERSION -rw---. 1 dirsrv dirsrv 303104 Nov 8 15:52 entryrdn.db -rw---. 1 dirsrv dirsrv 40960 Nov 8 15:52 entryusn.db -rw---. 1 dirsrv dirsrv 1523712 Nov 8 15:52 id2entry.db -rw---. 1 dirsrv dirsrv 90112 Nov 8 15:52 nsuniqueid.db -rw---. 1 dirsrv dirsrv 16384 Nov 8 15:52 numsubordinates.db -rw---. 1 dirsrv dirsrv 90112 Nov 8 15:52 objectclass.db -rw---. 1 dirsrv dirsrv 40960 Nov 8 15:52 parentid.db -rw---. 1 dirsrv dirsrv 16384 Nov 8 00:02 seeAlso.db -rw---. 1 dirsrv dirsrv 65536 Nov 8 15:52 targetuniqueid.db it's centOS 7 with IPA ipa-server-4.2.0-15.0.1.el7.centos.19.x86_64 >> >>What about the access logs? Do you see the CSN there? Did you check the DS access logs?? -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] CSN not found
On 11/03/2016 12:49 PM, lejeczek wrote: > > > On 03/11/16 14:16, Mark Reynolds wrote: >> >> On 11/03/2016 09:42 AM, lejeczek wrote: >>> hi everybody >>> >>> my three IPAs have gone haywire, two things I recall: one - one server >>> was on ScientificL with slightly lower minor version of IPA, two - >>> another server (of the two identical CEntOSes) had skewed time. >>> Not all there servers are in time-sync and all run same version of IPA > here I meant: Now all there >>> but replication broke with errors like: >>> >>> >>> $ ipa-replica-manage re-initialize --from rider --force >>> >>> .. >>> [03/Nov/2016:13:21:08 +] NSACLPlugin - The ACL target >>> cn=casigningcert >>> cert-pki-ca,cn=ca_renewal,cn=ipa,cn=etc,dc=dc=xx,dc=xx,dc=dc=xx,dc=xx,dc=x >>> >>> does not exist >>> [03/Nov/2016:13:21:08 +] NSACLPlugin - The ACL target >>> cn=casigningcert >>> cert-pki-ca,cn=ca_renewal,cn=ipa,cn=etc,dc=dc=xx,dc=xx,dc=dc=xx,dc=xx,dc=x >>> >>> does not exist >>> [03/Nov/2016:13:21:09 +] agmt="cn=meToswir.xx.xx.xx.xx.x" >>> (swir:389) - Can't locate CSN 581b120f00050004 in the changelog >>> (DB rc=-30988). If replication stops, the consumer may need to be >>> reinitialized. >>> [03/Nov/2016:13:21:09 +] NSMMReplicationPlugin - changelog program >>> - agmt="cn=meToswir.xx.xx.xx.xx.x" (swir:389): CSN >>> 581b120f00050004 not found, we aren't as up to date, or we purged >>> [03/Nov/2016:13:21:09 +] NSMMReplicationPlugin - >>> agmt="cn=meToswir.xx.xx.xx.xx.x" (swir:389): Data required to update >>> replica has been purged. The replica must be reinitialized. >>> [03/Nov/2016:13:21:09 +] NSMMReplicationPlugin - >>> agmt="cn=meToswir.xx.xx.xx.xx.x" (swir:389): Incremental update failed >>> and requires administrator action >>> >>> I did dbscan -f /var.../cb941db on all three servers and greped >>> but cannot see that 581b120f00050004 >>> >>> where to troubleshoot? >> What version of 389 do you have: >> >> rpm -qa | grep 389-ds-base >> >> Did you check the changelog database for 581b120f00050004: >> >> dbscan -f /var/lib/dirsrv/slapd-INSTANCE/db/changelogdb > results of above scan do not look like that CSN form reported in > dirsrv's error log, it is: > .. > =116156 > =116157 > =116158 > .. That doesn't look quite right, Just to confirm you should be doing something like dbscan -f /var/lib/dirsrv/slapd-master_1/db/changelogdb/fe665489-a13011e6-acbab8c1-43b12a38_581a3c410001.db | grep 581b120f00050004 >> >> What about the access logs? Do you see the CSN there? Did you check the DS access logs?? >> >> I've seen this issue before where a CSN is missing, which breaks the >> replication agreements, but the CSN does get added to the changelog >> after a few seconds. The only way to fix replication is to restart the >> server, or disable/enable the replication agreements(basically restart >> them). > restarting is not possible for the systemctl start ipa fails, though > system start dirsrv@... succeeds I meant restart the directory server, not freeipa: # restart-dirsrv > what would be correct process of removing repl agreements? You don't delete them, you just disable and re-enable them: https://access.redhat.com/documentation/en-US/Red_Hat_Directory_Server/10.1/html/Administration_Guide/disabling-replication.html > I'm trying disconnect/del but am not sure if this is the way. > >> Thanks, >> Mark >>> many thanks. >>> L >>> > -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] CSN not found
On 03/11/16 14:16, Mark Reynolds wrote: On 11/03/2016 09:42 AM, lejeczek wrote: hi everybody my three IPAs have gone haywire, two things I recall: one - one server was on ScientificL with slightly lower minor version of IPA, two - another server (of the two identical CEntOSes) had skewed time. Not all there servers are in time-sync and all run same version of IPA here I meant: Now all there but replication broke with errors like: $ ipa-replica-manage re-initialize --from rider --force .. [03/Nov/2016:13:21:08 +] NSACLPlugin - The ACL target cn=casigningcert cert-pki-ca,cn=ca_renewal,cn=ipa,cn=etc,dc=dc=xx,dc=xx,dc=dc=xx,dc=xx,dc=x does not exist [03/Nov/2016:13:21:08 +] NSACLPlugin - The ACL target cn=casigningcert cert-pki-ca,cn=ca_renewal,cn=ipa,cn=etc,dc=dc=xx,dc=xx,dc=dc=xx,dc=xx,dc=x does not exist [03/Nov/2016:13:21:09 +] agmt="cn=meToswir.xx.xx.xx.xx.x" (swir:389) - Can't locate CSN 581b120f00050004 in the changelog (DB rc=-30988). If replication stops, the consumer may need to be reinitialized. [03/Nov/2016:13:21:09 +] NSMMReplicationPlugin - changelog program - agmt="cn=meToswir.xx.xx.xx.xx.x" (swir:389): CSN 581b120f00050004 not found, we aren't as up to date, or we purged [03/Nov/2016:13:21:09 +] NSMMReplicationPlugin - agmt="cn=meToswir.xx.xx.xx.xx.x" (swir:389): Data required to update replica has been purged. The replica must be reinitialized. [03/Nov/2016:13:21:09 +] NSMMReplicationPlugin - agmt="cn=meToswir.xx.xx.xx.xx.x" (swir:389): Incremental update failed and requires administrator action I did dbscan -f /var.../cb941db on all three servers and greped but cannot see that 581b120f00050004 where to troubleshoot? What version of 389 do you have: rpm -qa | grep 389-ds-base Did you check the changelog database for 581b120f00050004: dbscan -f /var/lib/dirsrv/slapd-INSTANCE/db/changelogdb results of above scan do not look like that CSN form reported in dirsrv's error log, it is: .. =116156 =116157 =116158 .. What about the access logs? Do you see the CSN there? I've seen this issue before where a CSN is missing, which breaks the replication agreements, but the CSN does get added to the changelog after a few seconds. The only way to fix replication is to restart the server, or disable/enable the replication agreements(basically restart them). restarting is not possible for the systemctl start ipa fails, though system start dirsrv@... succeeds what would be correct process of removing repl agreements? I'm trying disconnect/del but am not sure if this is the way. Thanks, Mark many thanks. L -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] CSN not found
On 11/03/2016 09:42 AM, lejeczek wrote: > hi everybody > > my three IPAs have gone haywire, two things I recall: one - one server > was on ScientificL with slightly lower minor version of IPA, two - > another server (of the two identical CEntOSes) had skewed time. > Not all there servers are in time-sync and all run same version of IPA > but replication broke with errors like: > > > $ ipa-replica-manage re-initialize --from rider --force > > .. > [03/Nov/2016:13:21:08 +] NSACLPlugin - The ACL target > cn=casigningcert > cert-pki-ca,cn=ca_renewal,cn=ipa,cn=etc,dc=dc=xx,dc=xx,dc=dc=xx,dc=xx,dc=x > does not exist > [03/Nov/2016:13:21:08 +] NSACLPlugin - The ACL target > cn=casigningcert > cert-pki-ca,cn=ca_renewal,cn=ipa,cn=etc,dc=dc=xx,dc=xx,dc=dc=xx,dc=xx,dc=x > does not exist > [03/Nov/2016:13:21:09 +] agmt="cn=meToswir.xx.xx.xx.xx.x" > (swir:389) - Can't locate CSN 581b120f00050004 in the changelog > (DB rc=-30988). If replication stops, the consumer may need to be > reinitialized. > [03/Nov/2016:13:21:09 +] NSMMReplicationPlugin - changelog program > - agmt="cn=meToswir.xx.xx.xx.xx.x" (swir:389): CSN > 581b120f00050004 not found, we aren't as up to date, or we purged > [03/Nov/2016:13:21:09 +] NSMMReplicationPlugin - > agmt="cn=meToswir.xx.xx.xx.xx.x" (swir:389): Data required to update > replica has been purged. The replica must be reinitialized. > [03/Nov/2016:13:21:09 +] NSMMReplicationPlugin - > agmt="cn=meToswir.xx.xx.xx.xx.x" (swir:389): Incremental update failed > and requires administrator action > > I did dbscan -f /var.../cb941db on all three servers and greped > but cannot see that 581b120f00050004 > > where to troubleshoot? What version of 389 do you have: rpm -qa | grep 389-ds-base Did you check the changelog database for 581b120f00050004: dbscan -f /var/lib/dirsrv/slapd-INSTANCE/db/changelogdb What about the access logs? Do you see the CSN there? I've seen this issue before where a CSN is missing, which breaks the replication agreements, but the CSN does get added to the changelog after a few seconds. The only way to fix replication is to restart the server, or disable/enable the replication agreements(basically restart them). Thanks, Mark > many thanks. > L > -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
[Freeipa-users] CSN not found
hi everybody my three IPAs have gone haywire, two things I recall: one - one server was on ScientificL with slightly lower minor version of IPA, two - another server (of the two identical CEntOSes) had skewed time. Not all there servers are in time-sync and all run same version of IPA but replication broke with errors like: $ ipa-replica-manage re-initialize --from rider --force .. [03/Nov/2016:13:21:08 +] NSACLPlugin - The ACL target cn=casigningcert cert-pki-ca,cn=ca_renewal,cn=ipa,cn=etc,dc=dc=xx,dc=xx,dc=dc=xx,dc=xx,dc=x does not exist [03/Nov/2016:13:21:08 +] NSACLPlugin - The ACL target cn=casigningcert cert-pki-ca,cn=ca_renewal,cn=ipa,cn=etc,dc=dc=xx,dc=xx,dc=dc=xx,dc=xx,dc=x does not exist [03/Nov/2016:13:21:09 +] agmt="cn=meToswir.xx.xx.xx.xx.x" (swir:389) - Can't locate CSN 581b120f00050004 in the changelog (DB rc=-30988). If replication stops, the consumer may need to be reinitialized. [03/Nov/2016:13:21:09 +] NSMMReplicationPlugin - changelog program - agmt="cn=meToswir.xx.xx.xx.xx.x" (swir:389): CSN 581b120f00050004 not found, we aren't as up to date, or we purged [03/Nov/2016:13:21:09 +] NSMMReplicationPlugin - agmt="cn=meToswir.xx.xx.xx.xx.x" (swir:389): Data required to update replica has been purged. The replica must be reinitialized. [03/Nov/2016:13:21:09 +] NSMMReplicationPlugin - agmt="cn=meToswir.xx.xx.xx.xx.x" (swir:389): Incremental update failed and requires administrator action I did dbscan -f /var.../cb941db on all three servers and greped but cannot see that 581b120f00050004 where to troubleshoot? many thanks. L -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
