Hello world,
I have 3 IPA servers that are supposed to all replicate with each other. For
one server this stopped working. On all servers I have ipa-server
4.12.2-14.el9_6 on Rocky Linux 9.6.
I'll call my servers A, B, and C. Server A cannot replicate with neither server
B nor C. B and C can replicate eachother without issues. My first suspicion was
a firewall, but I've already confirmed that that's not the issue.
Here's the sanitized CLI outputs, all from server A:
```
$ ipa-healthcheck
{
"source": "ipahealthcheck.ds.replication",
"check": "ReplicationCheck",
"result": "CRITICAL",
"uuid": "9c96a61a-d917-44de-907a-d5c7a4df667e",
"when": "20250606095130Z",
"duration": "1.072988",
"kw": {
"key": "DSREPLLE0001",
"items": [
"Replication",
"Agreement"
],
"msg": "The replication agreement (server-A.example.com-to-server-C.example.com) under
\"dc=example,dc=com\" is not in synchronization."
}
},
{
"source": "ipahealthcheck.ds.replication",
"check": "ReplicationCheck",
"result": "CRITICAL",
"uuid": "6c29d38e-a216-414e-b47a-d9302f38da67",
"when": "20250606095131Z",
"duration": "1.073047",
"kw": {
"key": "DSREPLLE0001",
"items": [
"Replication",
"Agreement"
],
"msg": "The replication agreement (metoserver-B.example.com) under
\"dc=example,dc=com\" is not in synchronization."
}
},
{
"source": "ipahealthcheck.ds.replication",
"check": "ReplicationCheck",
"result": "CRITICAL",
"uuid": "445f686b-27d9-47e0-aaa8-bb8d5f3416d4",
"when": "20250606095131Z",
"duration": "1.073054",
"kw": {
"key": "DSREPLLE0001",
"items": [
"Replication",
"Agreement"
],
"msg": "The replication agreement (catoserver-B.example.com) under
\"o=ipaca\" is not in synchronization."
}
},
{
"source": "ipahealthcheck.ds.replication",
"check": "ReplicationCheck",
"result": "CRITICAL",
"uuid": "bb6fb4b1-54fc-45dc-bb33-c8c3562f5765",
"when": "20250606095131Z",
"duration": "1.073058",
"kw": {
"key": "DSREPLLE0001",
"items": [
"Replication",
"Agreement"
],
"msg": "The replication agreement (server-A.example.com-to-server-C.example.com) under
\"o=ipaca\" is not in synchronization."
}
}
```
```
$ ipa-replica-manage list
ipa-replica-manage list
Directory Manager password:
server-A.example.com: master
server-B.example.com: master
server-C.example.com: master
```
Ok, things are broken, let's try force-sync.
```
$ ipa-replica-manage force-sync --from server-B.example.com --verbose --debug
<snip imports>
ipa: DEBUG: found 1 A records for server-A.example.com.: 192.168.12.58
ipa: DEBUG: The DNS response does not contain an answer to the question:
server-A.example.com. IN AAAA
Directory Manager password:
ipa: DEBUG: Created connection context.ldap2_139632322538896
ipa: DEBUG: found 1 A records for server-A.example.com.: 192.168.12.58
ipa: DEBUG: The DNS response does not contain an answer to the question:
server-A.example.com. IN AAAA
ipa: DEBUG: found 1 A records for server-B.example.com.: 192.168.13.128
ipa: DEBUG: The DNS response does not contain an answer to the question:
server-B.example.com. IN AAAA
ipa: DEBUG: retrieving schema for SchemaCache url=ldaps://server-A.example.com:636
conn=<ldap.ldapobject.SimpleLDAPObject object at 0x7efeaefd2460>
ipa: DEBUG: Loading StateFile from '/var/lib/ipa/sysrestore/sysrestore.state'
ipa: DEBUG: Loading Index file from '/var/lib/ipa/sysrestore/sysrestore.index'
ipa: DEBUG: retrieving schema for SchemaCache
url=ldapi://%2Frun%2Fslapd-BIN-BIOINF-NL.socket
conn=<ldap.ldapobject.SimpleLDAPObject object at 0x7efeaefd2610>
ipa: DEBUG: retrieving schema for SchemaCache url=ldaps://server-B.example.com:636
conn=<ldap.ldapobject.SimpleLDAPObject object at 0x7efeae1ee1f0>
ipa: INFO: Setting agreement
cn=meToserver-A.example.com,cn=replica,cn=dc\=example\,dc\=com,cn=mapping
tree,cn=config schedule to 2358-2359 0 to force synch
ipa: INFO: Deleting schedule 2358-2359 0 from agreement
cn=meToserver-A.example.com,cn=replica,cn=dc\=example\,dc\=com,cn=mapping
tree,cn=config
ipa: INFO: Replication Update in progress: False: status: Error (49) Problem
connecting to replica - LDAP error: Invalid credentials (connection error):
start: 19700101000000: end: 19700101000000
ipa: DEBUG: Destroyed connection context.ldap2_139632322538896
```
Maybe I can re-initialize?
```
$ ipa-replica-manage re-initialize --from server-B.example.com --verbose --debug
<snip import>
<snip DNS responses, same as above>
Directory Manager password:
ipa: DEBUG: Created connection context.ldap2_140512488991280
ipa: DEBUG: retrieving schema for SchemaCache url=ldaps://server-A.example.com:636
conn=<ldap.ldapobject.SimpleLDAPObject object at 0x7fcb9c310dc0>
ipa: DEBUG: retrieving schema for SchemaCache url=ldaps://server-B.example.com:636
conn=<ldap.ldapobject.SimpleLDAPObject object at 0x7fcb9c1e10a0>
ipa: INFO: Setting agreement
cn=meToserver-A.example.com,cn=replica,cn=dc\=example\,dc\=com,cn=mapping
tree,cn=config schedule to 2358-2359 0 to force synch
ipa: INFO: Deleting schedule 2358-2359 0 from agreement
cn=meToserver-A.example.com,cn=replica,cn=dc\=example\,dc\=com,cn=mapping
tree,cn=config
Update in progress, 15 seconds elapsed
[ldaps://server-B.example.com:636] reports: Update failed! Status: [Error (49)
- LDAP error: Invalid credentials - no response received]
ipa: DEBUG: Destroyed connection context.ldap2_140512488991280
```
Is it a firewall or password issue? No:
```
$ ldapwhoami -H ldaps://server-B.example.com -D 'cn=directory manager' -W
Enter LDAP Password:
dn: cn=directory manager
```
From server-B:
```
server-B $ ldapwhoami -H ldaps://server-A.example.com -D 'cn=directory manager'
-W
Enter LDAP Password:
dn: cn=directory manager
```
I also checked that `ipa-replica-manage list-ruv` produces the same output for
all 3 servers.
Does anyone have any more ideas I could explore?
Many thanks in advance!
Peter
