Hi David, Did you ever resolve this?
I am literally having this issue as well in a 4 node cluster. I have completely blown away 3 replicas and reinitialized them, and have also tried lowering the replication time to 300s using dsconf but it still persists, no matter what I do. On Wed, Apr 16, 2025 at 11:52 AM David Brown via FreeIPA-users < [email protected]> wrote: > I agree; and that is usually what happens, but the fact that I still see > that error every few seconds on auth2 is what concerned me and why I > reached out to the group. > > I'm thinking there might be a deeper issue and I'm not sure how to figure > out what it is or what is causing it. > > Dave > > > > > > > On Wednesday, April 16, 2025 at 12:37:07 PM EDT, Mark Reynolds < > [email protected]> wrote: > > > > > > Well it shouldn't keep initializing the changelog. You should not see > it again after the message on 16/Apr/2025:10:50:31 > > Mark > > On 4/16/25 12:19 PM, David Brown wrote: > > I was able to create a user and it replicated and was able to delete > that same user from the secondary and it deleted it from the first. > > > > I just wasn't sure if something else was in error with those logs > continuing to error like that and say something was wrong with replication. > > > > Dave > > > > > > > > > > On Wednesday, April 16, 2025 at 12:04:55 PM EDT, Mark Reynolds < > [email protected]> wrote: > > > > > > > > > > > > > > Hi David, > > > > The logs look fine after the reinit. The last lines show the changelog > was successfully reinitialized (Rebuilding replication changelog RUV > complete. Result 0 (Success)) But if you have a doubt then make an update > on each replica and see if it's replicated to the other replica. > > > > HTH, > > > > Mark > > > > > > On 4/16/25 11:25 AM, David Brown via FreeIPA-users wrote: > > > > > >> > > Hi, > > > > > > > > > > I have a small two node FreeIPA setup. (auth1 & auth2) I noticed today > that I was getting a replication error on node 2 (auth2) about missing CSN > in the changelog. > > > > > > > > > > I reinitialized the two nodes replicating auth1 -> auth2 and this has > fixed replication issues in the past, but the error persists. > > > > > > > > > > I can create users and delete users from each side of the replication > and it appears to be replicating those changes and they seem > (non-definitively) to be in sync, but this error concerns me and > reinitializing doesn't appear to solve it. > > > > > > > > > > Here are the logs. > > > > > > > > > > Any help is resolving this would be fantastic as I'm not finding much > help via web searches. > > > > > > > > > > Thanks, David > > > > > > > > > > The sanitized error: > > > > > > > > > > Apr 16 10:50:31 auth2 ns-slapd[8419]: [16/Apr/2025:10:50:31.682835677 > -0400] - ERR - agmt="cn=caToauth1...." (auth1:389) - clcache_load_buffer - > Can't locate CSN 66587eaa000100050000 in the changelog (DB rc=-12797). If > replication stops, the consumer may need to be reinitialized. > > Apr 16 10:50:31 auth2 ns-slapd[8419]: [16/Apr/2025:10:50:31.684721395 > -0400] - ERR - NSMMReplicationPlugin - changelog program - > repl_plugin_name_cl - agmt="cn=caToauth1...." (auth1:389): CSN > 66587eaa000100050000 not found, we aren't as up to date, or we purged > > Apr 16 10:50:31 auth2 ns-slapd[8419]: [16/Apr/2025:10:50:31.685877312 > -0400] - ERR - NSMMReplicationPlugin - send_updates - > agmt="cn=caToauth1...." (auth1:389): Data required to update replica has > been purged from the changelog. If the error persists the replica must be > reinitialized. > > > > > > > > > > > > > > > > > > Santitized re-initialization > > > > > > > > > > ipa topologysegment-reinitialize domain auth1....-to-auth2.... --right > > > -------------------------------------------------------------------------------------------- > > Replication refresh for segment: "auth1....-to-auth2...." requested. > > > -------------------------------------------------------------------------------------------- > > > > > > > > > > > > > > > > > > The sanitized logs of the re-initialization > > > > > > > > > > Apr 16 10:45:59 auth2 ns-slapd[8419]: [16/Apr/2025:10:45:59.963872886 > -0400] - ERR - ipa-topology-plugin - ipa_topo_be_state_changebackend > userRoot is going offline; inactivate plugin > > Apr 16 10:45:59 auth2 ns-slapd[8419]: [16/Apr/2025:10:45:59.966877380 > -0400] - NOTICE - NSMMReplicationPlugin - multisupplier_be_state_change - > Replica dc=...,dc=... is going offline; disabling replication > > Apr 16 10:46:00 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:00.189646600 > -0400] - INFO - bdb_instance_start - Import is running with > nsslapd-db-private-import-mem on; No other process is allowed to access the > database > > Apr 16 10:46:00 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:00.891598813 > -0400] - ERR - agmt="cn=caToauth1...." (auth1:389) - clcache_load_buffer - > Can't locate CSN 66587eaa000100050000 in the changelog (DB rc=-12797). If > replication stops, the consumer may need to be reinitialized. > > Apr 16 10:46:00 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:00.893083190 > -0400] - ERR - NSMMReplicationPlugin - changelog program - > repl_plugin_name_cl - agmt="cn=caToauth1...." (auth1:389): CSN > 66587eaa000100050000 not found, we aren't as up to date, or we purged > > Apr 16 10:46:00 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:00.894538030 > -0400] - ERR - NSMMReplicationPlugin - send_updates - > agmt="cn=caToauth1...." (auth1:389): Data required to update replica has > been purged from the changelog. If the error persists the replica must be > reinitialized. > > Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.403074677 > -0400] - INFO - bdb_import_monitor_threads - import userRoot: Workers > finished; cleaning up... > > Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.605238770 > -0400] - INFO - bdb_import_monitor_threads - import userRoot: Workers > cleaned up. > > Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.606678188 > -0400] - INFO - bdb_public_bdb_import_main - import userRoot: Indexing > complete. Post-processing... > > Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.607860375 > -0400] - INFO - bdb_public_bdb_import_main - import userRoot: Generating > numsubordinates (this may take several minutes to complete)... > > Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.618231549 > -0400] - INFO - bdb_public_bdb_import_main - import userRoot: Generating > numSubordinates complete. > > Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.620014951 > -0400] - INFO - bdb_get_nonleaf_ids - import userRoot: Gathering ancestorid > non-leaf IDs... > > Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.621434375 > -0400] - INFO - bdb_get_nonleaf_ids - import userRoot: Finished gathering > ancestorid non-leaf IDs. > > Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.622574267 > -0400] - INFO - ldbm_get_nonleaf_ids - import userRoot: Starting sort of > ancestorid non-leaf IDs... > > Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.623757818 > -0400] - INFO - ldbm_get_nonleaf_ids - import userRoot: Finished sort of > ancestorid non-leaf IDs. > > Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.626748319 > -0400] - INFO - bdb_ancestorid_new_idl_create_index - import userRoot: > Creating ancestorid index (new idl)... > > Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.632623820 > -0400] - INFO - bdb_ancestorid_new_idl_create_index - import userRoot: > Created ancestorid index (new idl). > > Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.633896253 > -0400] - INFO - bdb_public_bdb_import_main - import userRoot: Flushing > caches... > > Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.635305588 > -0400] - INFO - bdb_public_bdb_import_main - import userRoot: Closing > files... > > Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.728196646 > -0400] - INFO - bdb_public_bdb_import_main - import userRoot: Import > complete. Processed 729 entries in 3 seconds. (243.00 entries/sec) > > Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.737959417 > -0400] - ERR - ipa-topology-plugin - ipa_topo_be_state_change - backend > userRoot is coming online; checking domain level and init shared topology > > Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.744152900 > -0400] - NOTICE - NSMMReplicationPlugin - multisupplier_be_state_change - > Replica dc=...,dc=... is coming online; enabling replication > > Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.754743353 > -0400] - WARN - NSMMReplicationPlugin - replica_reload_ruv - New data for > replica dc=...,dc=... does not match the data in the changelog. > > Apr 16 10:46:02 auth2 ns-slapd[8419]: Recreating the changelog file. > This could affect replication with replica's consumers in which case the > consumers should be reinitialized. > > Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.862749463 > -0400] - NOTICE - NSMMReplicationPlugin - changelog program - > _cl5ConstructRUVs - Rebuilding the replication changelog RUV, this may take > several minutes... > > Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.864263319 > -0400] - NOTICE - NSMMReplicationPlugin - changelog program - > _cl5ConstructRUVs - Rebuilding replication changelog RUV complete. Result > 0 (Success) > > Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.872479720 > -0400] - ERR - cos-plugin - cos_dn_defs_cb - Skipping CoS Definition > cn=Password Policy,cn=accounts,dc=...,dc=...--no CoS Templates found, which > should be added before the CoS Definition. > > Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.874025309 > -0400] - WARN - NSACLPlugin - acl_parse - The ACL target > cn=groups,cn=compat,dc=...,dc=... does not exist > > Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.875303781 > -0400] - WARN - NSACLPlugin - acl_parse - The ACL target > cn=computers,cn=compat,dc=...,dc=... does not exist > > Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.876489711 > -0400] - WARN - NSACLPlugin - acl_parse - The ACL target > cn=ng,cn=compat,dc=...,dc=... does not exist > > Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.877770904 > -0400] - WARN - NSACLPlugin - acl_parse - The ACL target > ou=sudoers,dc=...,dc=... does not exist > > Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.879231097 > -0400] - WARN - NSACLPlugin - acl_parse - The ACL target > cn=users,cn=compat,dc=...,dc=... does not exist > > Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.880458410 > -0400] - WARN - NSACLPlugin - acl_parse - The ACL target > cn=vaults,cn=kra,dc=...,dc=... does not exist > > Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.881648891 > -0400] - WARN - NSACLPlugin - acl_parse - The ACL target > cn=vaults,cn=kra,dc=...,dc=... does not exist > > Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.882722133 > -0400] - WARN - NSACLPlugin - acl_parse - The ACL target > cn=vaults,cn=kra,dc=...,dc=... does not exist > > Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.884124162 > -0400] - WARN - NSACLPlugin - acl_parse - The ACL target > cn=vaults,cn=kra,dc=...,dc=... does not exist > > Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.885222292 > -0400] - WARN - NSACLPlugin - acl_parse - The ACL target > cn=vaults,cn=kra,dc=...,dc=... does not exist > > Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.886404863 > -0400] - WARN - NSACLPlugin - acl_parse - The ACL target > cn=vaults,cn=kra,dc=...,dc=... does not exist > > Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.887615474 > -0400] - WARN - NSACLPlugin - acl_parse - The ACL target > cn=vaults,cn=kra,dc=...,dc=... does not exist > > Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.889102423 > -0400] - WARN - NSACLPlugin - acl_parse - The ACL target > cn=vaults,cn=kra,dc=...,dc=... does not exist > > Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.890327963 > -0400] - WARN - NSACLPlugin - acl_parse - The ACL target > cn=vaults,cn=kra,dc=...,dc=... does not exist > > Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.891412886 > -0400] - WARN - NSACLPlugin - acl_parse - The ACL target > cn=vaults,cn=kra,dc=...,dc=... does not exist > > Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.892586141 > -0400] - WARN - NSACLPlugin - acl_parse - The ACL target > cn=vaults,cn=kra,dc=...,dc=... does not exist > > Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.899706161 > -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=casigningcert > cert-pki-ca,cn=ca_renewal,cn=ipa,cn=etc,dc=...,dc=... does not exist > > Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.901020418 > -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=casigningcert > cert-pki-ca,cn=ca_renewal,cn=ipa,cn=etc,dc=...,dc=... does not exist > > Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.907037194 > -0400] - NOTICE - NSMMReplicationPlugin - changelog program - > _cl5ConstructRUVs - Rebuilding the replication changelog RUV, this may take > several minutes... > > Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.908357262 > -0400] - NOTICE - NSMMReplicationPlugin - changelog program - > _cl5ConstructRUVs - Rebuilding replication changelog RUV complete. Result > 0 (Success) > > > > > > > > > > > -- > Identity Management Development Team > > -- > _______________________________________________ > FreeIPA-users mailing list -- [email protected] > To unsubscribe send an email to [email protected] > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedorahosted.org/archives/list/[email protected] > Do not reply to spam, report it: > https://pagure.io/fedora-infrastructure/new_issue >
-- _______________________________________________ FreeIPA-users mailing list -- [email protected] To unsubscribe send an email to [email protected] Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/[email protected] Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
