On 05/06/2015 04:25 PM, thierry bordaz wrote:
On 05/06/2015 03:19 PM, Martin Babinsky wrote:
replies are inline.
On 05/06/2015 02:22 PM, thierry bordaz wrote:
On 05/06/2015 01:54 PM, Martin Babinsky wrote:
The attached patch tries to fix
After discussion with Thierry we concluded that while this issue is
more complex than it seems, the transition from REPLACE to DEL/ADD
operations when updating nsDS5ReplicaId should suffice for this ticket.
Few comments, you are using MOD_DEL 'replicaID' with None value. So this
is going to delete all previous values and it should be equivalent to a
I was thinking you wanted to retrieve the id_value and call MOD_DEL
'replicaID' <current_value>. So that if by the time you fetched the
replicaId, an other replica updated the replicaId, the MOD_DEL/MOD_ADD
would fail and you need a new iteration.
Sorry I didn't know you can MOD_DEL a particular value (I'm LDAP noob
I know). Will fix this.
If replicaId was multi-valued and you want to make it single valued, you
may want to do create a more complex MOD (e.g. (ldap.MOD_DELETE,
'nsDS5ReplicaId', str(value1), (ldap.MOD_DELETE, 'nsDS5ReplicaId',
AFAIK ReplicaId is single-valued (looking at the schema right now) so
this shouldn't be problem.
If it is updating successfully do you want to return 'retval' or
If several replicas try to update the replicaId of the master and the
current replicaId is 1000.
Replica1 successfully updates the replicaId and gets 1001 as the new
Replica2 successfully updates the replicaId and gets 1002.
The final value on master will be 1002, but replica1 will assum it is
1001. Is it a problem ?
I studied the code in the master branch and IIUC (and please correct
me if I got this wrong) nsDS5ReplicaId attribute in
'cn=replication,cn=etc,$SUFFIX' represents replicaID of the _next_
replica that will be installed.
So if a replica is installed, it sets the current value of
nsDS5ReplicaId as its replica ID (the function returns 'retval') and
then increments it in 'cn=replication,cn=etc,$SUFFIX' entry ('retval +
1' is written to master) so that the next installed replica fetches
this updated value.
So the case you described should be the expected behavior. To change
it would require different patch IMHO.
Thank for your precious explanations, in fact the value in
'cn=replication,cn=etc,SUFFIX' is the next available replicaId value and
is the centralized mechanism that assign unique replicaID.
The risk was that several replica pick the same value. So yes it is
important that the MOD_DEL specifies the previously read value so that
the test/set will be atomic. If several replicas read the same value,
only the faster one will use it to install the replica.
Attaching updated patch with fixed MOD_DELETE operation.
From bdb686454ae7a7066ea5b568b91fcf88ba78d4b8 Mon Sep 17 00:00:00 2001
From: Martin Babinsky <mbabi...@redhat.com>
Date: Mon, 4 May 2015 18:33:44 +0200
Subject: [PATCH] prevent duplicate IDs when setting up multiple replicas
against single master
This patch forces replicas to use DELETE+ADD operations to increment
'nsDS5ReplicaId' in 'cn=replication,cn=etc,$SUFFIX' on master, and retry
multiple times in the case of conflict with another update. Thus when multiple
replicas are set-up against single master none of them will have duplicate ID.
ipaserver/install/replication.py | 78 ++++++++++++++++++++++++++--------------
1 file changed, 52 insertions(+), 26 deletions(-)
diff --git a/ipaserver/install/replication.py b/ipaserver/install/replication.py
index 66764c22f69328942fe2e4581cfafb3806438d7c..57d4b9917781e5d459856480050c8ce1457c25dd 100644
@@ -21,6 +21,7 @@ import time
+from random import randint
@@ -230,34 +231,59 @@ class ReplicationManager(object):
# Ok, either the entry doesn't exist or the attribute isn't set
# so get it from the other master
- retval = -1
+ return self._get_and_update_id_from_master(master_conn)
+ def _get_and_update_id_from_master(self, master_conn, attempts=5):
+ Fetch replica ID from remote master and update nsDS5ReplicaId attribute
+ on 'cn=replication,cn=etc,$SUFFIX' entry. Do it as MOD_DELETE+MOD_ADD
+ operations and retry when conflict occurs, e.g. due to simultaneous
+ update from another replica.
+ :param master_conn: LDAP connection to master
+ :param attempts: number of attempts to update nsDS5ReplicaId
+ :return: value of nsDS5ReplicaId before incrementation
dn = DN(('cn','replication'),('cn','etc'), self.suffix)
- replica = master_conn.get_entry(dn)
- except errors.NotFound:
- root_logger.debug("Unable to retrieve nsDS5ReplicaId from remote server")
- id_values = replica.get('nsDS5ReplicaId')
- if not id_values:
+ for a in range(1, attempts + 1):
+ root_logger.debug('Fetching nsDS5ReplicaId from master '
+ '[attempt %d/%d]', a, attempts)
+ replica = master_conn.get_entry(dn)
+ id_values = replica.get('nsDS5ReplicaId')
+ if not id_values:
+ root_logger.debug("Unable to retrieve nsDS5ReplicaId from remote server")
+ raise RuntimeError("Unable to retrieve nsDS5ReplicaId from remote server")
+ # nsDS5ReplicaId is single-valued now, but historically it could
+ # contain multiple values, of which we need the highest.
+ # see bug: https://fedorahosted.org/freeipa/ticket/3394
+ retval = max(int(v) for v in id_values)
+ # Now update the value on the master
+ mod_list = [(ldap.MOD_DELETE, 'nsDS5ReplicaId', retval),
+ (ldap.MOD_ADD, 'nsDS5ReplicaId', str(retval + 1))]
+ master_conn.modify_s(dn, mod_list)
+ root_logger.debug('Successfully updated nsDS5ReplicaId.')
+ return retval
+ except errors.NotFound:
root_logger.debug("Unable to retrieve nsDS5ReplicaId from remote server")
- raise RuntimeError("Unable to retrieve nsDS5ReplicaId from remote server")
- # nsDS5ReplicaId is single-valued now, but historically it could
- # contain multiple values, of which we need the highest.
- # see bug: https://fedorahosted.org/freeipa/ticket/3394
- retval = max(int(v) for v in id_values)
- # Now update the value on the master
- mod = [(ldap.MOD_REPLACE, 'nsDS5ReplicaId', str(retval + 1))]
- master_conn.modify_s(dn, mod)
- except Exception, e:
- root_logger.debug("Problem updating nsDS5ReplicaID %s" % e)
- return retval
+ # these errors signal a conflict in updating replica ID.
+ # We then wait for a random time interval and try again
+ except (ldap.NO_SUCH_ATTRIBUTE, ldap.OBJECT_CLASS_VIOLATION) as e:
+ sleep_interval = randint(1, 5)
+ root_logger.debug("Update failed (%s). Conflicting operation?",
+ # in case of other error we bail out
+ except ldap.LDAPError as e:
+ root_logger.debug("Problem updating nsDS5ReplicaID %s" % e)
+ raise RuntimeError("Failed to update nsDS5ReplicaId in %d attempts"
+ % attempts)
def get_agreement_filter(self, agreement_types=None, host=None):
Manage your subscription for the Freeipa-devel mailing list:
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code