On Fri, Jun 19, 2015 at 03:18:50PM -0400, Rob Crittenden wrote:
Rich Megginson wrote:
On 06/19/2015 12:22 PM, Andrew E. Bruno wrote:
Questions:
0. Is it likely that after running out of file descriptors the dirsrv
slapd database on rep2 was corrupted?
That would appear to be the case based on correlation of events,
although I've never seen that happen, and it is not supposed to happen.
1. Do we have to run ipa-replica-manage del rep2 on *each* of the
remaining replica servers (rep1 and rep3)? Or should it just be run on
the first master?
I believe it should only be run on the first master, but it hung, so
something is not right, and I'm not sure how to remedy the situation.
How long did it hang, and where?
This command was run on rep1 (first master):
[rep1]$ ipa-replica-manage del rep2
This command hung.. (~10 minutes..) until I Ctr-C. After noticing ldap
queries were hanging on rep2 we ran this on rep2:
[rep2]$ systemctl stop ipa
(shutdown all ipa services on rep2)
Then back on rep1 (first master)
[rep1]$ ipa-replica-manage -v --force del rep2
Which appeared to work ok.
Do we need to run ipa-csreplicate-manage del as well?
2. Why does the rep2 server still appear when querying the
nsDS5ReplicationAgreement in ldap? Is this benign or will this pose
problems
when we go to add rep2 back in?
You should remove it.
And ipa-csreplica-manage is the tool to do it.
When I run this on rep1 (first master):
[rep1]$ ipa-csreplica-manage list
Directory Manager password:
rep3: master
rep1: master
[rep1]$ ipa-csreplica-manage del rep2
Directory Manager password:
'rep1' has no replication agreement for 'rep2'
But seems to still be there:
[rep1]$ ldapsearch -Y GSSAPI -b "cn=mapping tree,cn=config"
objectClass=nsDS5ReplicationAgreement -LL
dn: cn=masterAgreement1-rep3-pki-tomcat,cn=replica,cn=ipaca,cn=mapping
tree,cn=config
objectClass: top
objectClass: nsds5replicationagreement
cn: masterAgreement1-rep3-pki-tomcat
nsDS5ReplicaRoot: o=ipaca
nsDS5ReplicaHost: rep3
nsDS5ReplicaPort: 389
nsDS5ReplicaBindDN: cn=Replication Manager
cloneAgreement1-rep3-pki-tomcat,ou=csusers,cn=config
nsDS5ReplicaBindMethod: Simple
nsDS5ReplicaTransportInfo: TLS
description: masterAgreement1-rep3-pki-tomcat
nsds50ruv: {replicageneration} 5527f74b000000600000
nsds50ruv: {replica 91 ldap://rep3:389} 5537c7ba0000005b
0000 5582c7e40004005b0000
nsds50ruv: {replica 96 ldap://rep1:389} 5527f75400000060
0000 5582cd19000000600000
nsds50ruv: {replica 97 ldap://rep2:389} 5527f76000000061
0000 556f462b000400610000
nsruvReplicaLastModified: {replica 91 ldap://rep3:389} 0
0000000
nsruvReplicaLastModified: {replica 96 ldap://rep1:389} 0
0000000
nsruvReplicaLastModified: {replica 97 ldap://rep2:389} 0
0000000
nsds5replicaLastUpdateStart: 20150619193149Z
nsds5replicaLastUpdateEnd: 20150619193149Z
nsds5replicaChangesSentSinceStartup:: OTY6MTMyLzAg
nsds5replicaLastUpdateStatus: 0 Replica acquired successfully: Incremental upd
ate succeeded
nsds5replicaUpdateInProgress: FALSE
nsds5replicaLastInitStart: 0
nsds5replicaLastInitEnd: 0
However, when I run the ldapsearch on rep3 it's not there (the
cn=ipaca,cn=mapping tree,cn=config is not listed):
[rep3]$ ldapsearch -Y GSSAPI -b "cn=mapping tree,cn=config"
objectClass=nsDS5ReplicationAgreement -LL
dn: cn=meTorep1,cn=replica,cn=dc\3Dccr\2Cdc\3Dbuffalo\2C dc\3Dedu,cn=mapping
tree,cn=config
cn: meTorep1
objectClass: nsds5replicationagreement
objectClass: top
nsDS5ReplicaTransportInfo: LDAP
description: me to rep1
nsDS5ReplicaRoot: dc=ccr,dc=buffalo,dc=edu
nsDS5ReplicaHost: rep1
3. What steps/commands can we take to verify rep2 was successfully
removed and
replication is behaving normally?
The ldapsearch you performed already will confirm that the CA agreement has
been removed.
Still showing up.. Any thoughts?
At this point we want to ensure both remaining masters are functional and
operating normally. Any other commands you recommend running to check?
