Hi, On Mon, Jun 16, 2025 at 9:10 AM Kroon PC, Peter via FreeIPA-users < freeipa-users@lists.fedorahosted.org> wrote:
> Hi Mark, > > thanks for chipping in. > Does anyone else have any suggestions before I wipe and reinstall the > replica entirely? > > You need to check if the credentials used for the replication are working. On the first server: [server1]# kinit -kt /etc/dirsrv/ds.keytab ldap/server1.ipa.test [server1]# ldapsearch -Y GSSAPI -h server1.ipa.test -b "" -s base [server1]# ldapsearch -Y GSSAPI -h server2.ipa.test -b "" -s base The first LDAP search authenticates to the local LDAP server with the credentials from ds keytab, the 2nd LDAP search authenticates to the other master. Both should succeed. If one fails, you need to check the kvno: [server1]# klist -kte /etc/dirsrv/ds.keytab [server1]# kvno ldap/server1.ipa.t...@ipa.test The klist command outputs the kvno stored in the ds.keytab file. It has to match the kvno returned by the 2nd command (the kvno as seen by the kerberos server). If they don't match, it means that the key has been re-generated but there is an inconsistency between the one stored in the ds.keytab file and the one stored by the kerberos server. Also check that you get the same value for the KVNO on the other master [server2]# kadmin.local getprinc ldap/server.ipa.t...@ipa.test The above command directly checks in the kerberos server. The repair steps depend on which key is the most recent one, and where it is stored. Please provide the kvnos from all the commands and we'll try to guide you. flo Peter > > ________________________________________ > From: Mark Reynolds <marey...@redhat.com> > Sent: Friday, 6 June 2025 17:32 > To: FreeIPA users list > Cc: Kroon PC, Peter > Subject: Re: [Freeipa-users] Re: Replication issues > > [You don't often get email from marey...@redhat.com. Learn why this is > important at https://aka.ms/LearnAboutSenderIdentification ] > > On 6/6/25 11:01 AM, Kroon PC, Peter via FreeIPA-users wrote: > > Hi Mark, > > > > thanks for the quick reply. > > Server-B has the following in the access log (twice, actually): > > ``` > > [06/Jun/2025:14:55:33.573567003 +0000] conn=8984 fd=372 slot=372 > connection from 192.168.12.58 to 192.168.13.128 > > [06/Jun/2025:14:55:33.613280772 +0000] conn=8984 op=0 BIND dn="" > method=sasl version=3 mech=GSSAPI > > [06/Jun/2025:14:55:33.614444924 +0000] conn=8984 op=0 RESULT err=49 > tag=97 nentries=0 wtime=0.000406209 optime=0.001176425 etime=0.001580230 - > SASL(-13): authentication failure: GSSAPI Failure: gss_accept_sec_context > > [06/Jun/2025:14:55:33.696712718 +0000] conn=8984 op=1 UNBIND > > [06/Jun/2025:14:55:33.696753716 +0000] conn=8984 op=1 fd=372 Disconnect > - Cleanly Closed Connection - U1 > > ``` > > I don't fully understand why it's trying GSSAPI after I entered the > directory manager password. > You authenticated as Directory Manager to the CLI, but it invokes the > replication agreement credentials to do the replication initialization > behind the scenes. > > I'd assume it'd use simple bind. > > In the security log there's the following, nothing about the failed > gssapi bind. > > ``` > > { "date": "[06\/Jun\/2025:14:55:00.776221689 +0000] ", "utc_time": > "1749221700.776221689", "event": "BIND_SUCCESS", "dn": "cn=directory > manager", "bind_method": "SIMPLE", "root_dn": true, "client_ip": > "192.168.12.58", "server_ip": "192.168.13.128", "ldap_version": 3, > "conn_id": 8983, "op_id": 0, "msg": "" } > > ``` > This is a different connection, but the other failed bind should have > been logged. Might have a bug somewhere :-( > > > > I'm about to leave the office for a long weekend, so I probably won't > respond again before Tuesday. > > So you "might" need to do an offline initialization to fix this, but I > suspect there is an official way the IPA team recommends to fix this > (sorry not as familiar with IPA as I am DS). So maybe someone from the > core IPA team and provide their recommendation. > > Mark > > > > > Peter > > > > ________________________________________ > > From: Mark Reynolds <marey...@redhat.com> > > Sent: Friday, 6 June 2025 15:26 > > To: FreeIPA users list > > Cc: Kroon PC, Peter > > Subject: Re: [Freeipa-users] Replication issues > > > > [You don't often get email from marey...@redhat.com. Learn why this is > important at https://aka.ms/LearnAboutSenderIdentification ] > > > > Hi Peter, > > > > So the credentials the replication agreement is using are not valid (for > > whatever reason). Please check the directory server access log for > > "err=49" and you should see the bind DN and more details on why the > > authentication failed. > > > > HTH, > > > > Mark > > > > On 6/6/25 6:56 AM, Kroon PC, Peter via FreeIPA-users wrote: > >> Hello world, > >> > >> I have 3 IPA servers that are supposed to all replicate with each > other. For one server this stopped working. On all servers I have > ipa-server 4.12.2-14.el9_6 on Rocky Linux 9.6. > >> I'll call my servers A, B, and C. Server A cannot replicate with > neither server B nor C. B and C can replicate eachother without issues. My > first suspicion was a firewall, but I've already confirmed that that's not > the issue. > >> Here's the sanitized CLI outputs, all from server A: > >> ``` > >> $ ipa-healthcheck > >> { > >> "source": "ipahealthcheck.ds.replication", > >> "check": "ReplicationCheck", > >> "result": "CRITICAL", > >> "uuid": "9c96a61a-d917-44de-907a-d5c7a4df667e", > >> "when": "20250606095130Z", > >> "duration": "1.072988", > >> "kw": { > >> "key": "DSREPLLE0001", > >> "items": [ > >> "Replication", > >> "Agreement" > >> ], > >> "msg": "The replication agreement ( > server-A.example.com-to-server-C.example.com) under \"dc=example,dc=com\" > is not in synchronization." > >> } > >> }, > >> { > >> "source": "ipahealthcheck.ds.replication", > >> "check": "ReplicationCheck", > >> "result": "CRITICAL", > >> "uuid": "6c29d38e-a216-414e-b47a-d9302f38da67", > >> "when": "20250606095131Z", > >> "duration": "1.073047", > >> "kw": { > >> "key": "DSREPLLE0001", > >> "items": [ > >> "Replication", > >> "Agreement" > >> ], > >> "msg": "The replication agreement (metoserver-B.example.com) > under \"dc=example,dc=com\" is not in synchronization." > >> } > >> }, > >> { > >> "source": "ipahealthcheck.ds.replication", > >> "check": "ReplicationCheck", > >> "result": "CRITICAL", > >> "uuid": "445f686b-27d9-47e0-aaa8-bb8d5f3416d4", > >> "when": "20250606095131Z", > >> "duration": "1.073054", > >> "kw": { > >> "key": "DSREPLLE0001", > >> "items": [ > >> "Replication", > >> "Agreement" > >> ], > >> "msg": "The replication agreement (catoserver-B.example.com) > under \"o=ipaca\" is not in synchronization." > >> } > >> }, > >> { > >> "source": "ipahealthcheck.ds.replication", > >> "check": "ReplicationCheck", > >> "result": "CRITICAL", > >> "uuid": "bb6fb4b1-54fc-45dc-bb33-c8c3562f5765", > >> "when": "20250606095131Z", > >> "duration": "1.073058", > >> "kw": { > >> "key": "DSREPLLE0001", > >> "items": [ > >> "Replication", > >> "Agreement" > >> ], > >> "msg": "The replication agreement ( > server-A.example.com-to-server-C.example.com) under \"o=ipaca\" is not in > synchronization." > >> } > >> } > >> ``` > >> ``` > >> $ ipa-replica-manage list > >> ipa-replica-manage list > >> Directory Manager password: > >> > >> server-A.example.com: master > >> server-B.example.com: master > >> server-C.example.com: master > >> ``` > >> Ok, things are broken, let's try force-sync. > >> ``` > >> $ ipa-replica-manage force-sync --from server-B.example.com --verbose > --debug > >> <snip imports> > >> ipa: DEBUG: found 1 A records for server-A.example.com.: 192.168.12.58 > >> ipa: DEBUG: The DNS response does not contain an answer to the > question: server-A.example.com. IN AAAA > >> Directory Manager password: > >> > >> ipa: DEBUG: Created connection context.ldap2_139632322538896 > >> ipa: DEBUG: found 1 A records for server-A.example.com.: 192.168.12.58 > >> ipa: DEBUG: The DNS response does not contain an answer to the > question: server-A.example.com. IN AAAA > >> ipa: DEBUG: found 1 A records for server-B.example.com.: 192.168.13.128 > >> ipa: DEBUG: The DNS response does not contain an answer to the > question: server-B.example.com. IN AAAA > >> ipa: DEBUG: retrieving schema for SchemaCache url=ldaps:// > server-A.example.com:636 conn=<ldap.ldapobject.SimpleLDAPObject object at > 0x7efeaefd2460> > >> ipa: DEBUG: Loading StateFile from > '/var/lib/ipa/sysrestore/sysrestore.state' > >> ipa: DEBUG: Loading Index file from > '/var/lib/ipa/sysrestore/sysrestore.index' > >> ipa: DEBUG: retrieving schema for SchemaCache > url=ldapi://%2Frun%2Fslapd-BIN-BIOINF-NL.socket > conn=<ldap.ldapobject.SimpleLDAPObject object at 0x7efeaefd2610> > >> ipa: DEBUG: retrieving schema for SchemaCache url=ldaps:// > server-B.example.com:636 conn=<ldap.ldapobject.SimpleLDAPObject object at > 0x7efeae1ee1f0> > >> ipa: INFO: Setting agreement > >> cn=meToserver-A.example.com,cn=replica,cn=dc\=example\,dc\=com,cn=mapping > tree,cn=config schedule to 2358-2359 0 to force synch > >> ipa: INFO: Deleting schedule 2358-2359 0 from agreement cn= > meToserver-A.example.com,cn=replica,cn=dc\=example\,dc\=com,cn=mapping > tree,cn=config > >> ipa: INFO: Replication Update in progress: False: status: Error (49) > Problem connecting to replica - LDAP error: Invalid credentials (connection > error): start: 19700101000000: end: 19700101000000 > >> ipa: DEBUG: Destroyed connection context.ldap2_139632322538896 > >> ``` > >> Maybe I can re-initialize? > >> ``` > >> $ ipa-replica-manage re-initialize --from server-B.example.com > --verbose --debug > >> <snip import> > >> <snip DNS responses, same as above> > >> Directory Manager password: > >> ipa: DEBUG: Created connection context.ldap2_140512488991280 > >> ipa: DEBUG: retrieving schema for SchemaCache url=ldaps:// > server-A.example.com:636 conn=<ldap.ldapobject.SimpleLDAPObject object at > 0x7fcb9c310dc0> > >> ipa: DEBUG: retrieving schema for SchemaCache url=ldaps:// > server-B.example.com:636 conn=<ldap.ldapobject.SimpleLDAPObject object at > 0x7fcb9c1e10a0> > >> ipa: INFO: Setting agreement > >> cn=meToserver-A.example.com,cn=replica,cn=dc\=example\,dc\=com,cn=mapping > tree,cn=config schedule to 2358-2359 0 to force synch > >> ipa: INFO: Deleting schedule 2358-2359 0 from agreement cn= > meToserver-A.example.com,cn=replica,cn=dc\=example\,dc\=com,cn=mapping > tree,cn=config > >> Update in progress, 15 seconds elapsed > >> [ldaps://server-B.example.com:636] reports: Update failed! Status: > [Error (49) - LDAP error: Invalid credentials - no response received] > >> ipa: DEBUG: Destroyed connection context.ldap2_140512488991280 > >> ``` > >> Is it a firewall or password issue? No: > >> ``` > >> $ ldapwhoami -H ldaps://server-B.example.com -D 'cn=directory manager' > -W > >> Enter LDAP Password: > >> dn: cn=directory manager > >> ``` > >> From server-B: > >> ``` > >> server-B $ ldapwhoami -H ldaps://server-A.example.com -D 'cn=directory > manager' -W > >> Enter LDAP Password: > >> dn: cn=directory manager > >> ``` > >> I also checked that `ipa-replica-manage list-ruv` produces the same > output for all 3 servers. > >> Does anyone have any more ideas I could explore? > >> > >> Many thanks in advance! > >> Peter > > -- > > Identity Management Development Team > > > -- > Identity Management Development Team > > -- > _______________________________________________ > FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org > To unsubscribe send an email to freeipa-users-le...@lists.fedorahosted.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org > Do not reply to spam, report it: > https://pagure.io/fedora-infrastructure/new_issue >
-- _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-le...@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
