Re: [Freeipa-devel] [PATCH 0377-0382] Synchronize changes from LDAP after reconnect

2015-06-02 Thread Tomas Hozza
On 05/28/2015 05:58 PM, Matus Honek wrote:
 Hi,

 functionality seems to work fine. I have not checked the code thoroughly.
 Kind of a test is attached (requires setting named's ldap connection 
 appropriately).

 ACK

 Matúš Honěk


 - Original Message -
 From: Petr Spacek pspa...@redhat.com
 To: tho...@redhat.com, Matus Honek mho...@redhat.com
 Cc: freeipa-devel@redhat.com
 Sent: Wednesday, May 27, 2015 2:50:52 PM
 Subject: [PATCH 0377-0382] Synchronize changes from LDAP after reconnect

 Hello,

 https://fedorahosted.org/bind-dyndb-ldap/ticket/128

 Previously records deleted when connection to LDAP server was down were not
 synchronized properly. It should work now.

 I use this command to simulate broken connections and connection 
 re-establishment:
 $ socat tcp-listen:3899,reuseaddr,fork tcp-connect:localhost:389

 It should be enough to add ldap://$(hostname):3899 as LDAP URI to
 /etc/named.conf and then simulate changes by killing and restarting socat.

 Let me know if you need any assistance!

Hi.

I did a formal review of the code. Everything looks good.

ACK.

Regards,

-- 
Tomas Hozza
Software Engineer - EMEA ENG Developer Experience

PGP: 1D9F3C2D
Red Hat Inc. http://cz.redhat.com

-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] [PATCH 0377-0382] Synchronize changes from LDAP after reconnect

2015-06-02 Thread Petr Spacek
On 2.6.2015 13:15, Tomas Hozza wrote:
 On 05/28/2015 05:58 PM, Matus Honek wrote:
 Hi,

 functionality seems to work fine. I have not checked the code thoroughly.
 Kind of a test is attached (requires setting named's ldap connection 
 appropriately).

 ACK

 Matúš Honěk


 - Original Message -
 From: Petr Spacek pspa...@redhat.com
 To: tho...@redhat.com, Matus Honek mho...@redhat.com
 Cc: freeipa-devel@redhat.com
 Sent: Wednesday, May 27, 2015 2:50:52 PM
 Subject: [PATCH 0377-0382] Synchronize changes from LDAP after reconnect

 Hello,

 https://fedorahosted.org/bind-dyndb-ldap/ticket/128

 Previously records deleted when connection to LDAP server was down were not
 synchronized properly. It should work now.

 I use this command to simulate broken connections and connection 
 re-establishment:
 $ socat tcp-listen:3899,reuseaddr,fork tcp-connect:localhost:389

 It should be enough to add ldap://$(hostname):3899 as LDAP URI to
 /etc/named.conf and then simulate changes by killing and restarting socat.

 Let me know if you need any assistance!

 Hi.
 
 I did a formal review of the code. Everything looks good.
 
 ACK.

Thank you very much!

Pushed to master:
9b4a6373c868f8858253d5e9bf850e1cbbed2a7f Avoid synchronization state resets.

783b04c87575205388a1277da8b46a781508f4a7 Consolidate synchronization state
machine to sync_state_change().

c727f40cae75b9f2e05f2789bade937c90202f11 On reconnect, detect and delete RBT
nodes which were removed from LDAP.

77ecee87f551567b94bd26290c734c7feb5ed93f Add iterators for dead nodes in 
metaLDAP.

b476041bd6a88b88cd1739e61960a666868e1b23 Increment MetaLDAP generation number
on reconnect.

57e87e325bbfe60709a53c8d5422339bb5f2b664 Add functions for MetaLDAP generation
number manipulation.

We are well on track to 8.0 release :-)

-- 
Petr^2 Spacek

-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] [PATCH 0377-0382] Synchronize changes from LDAP after reconnect

2015-05-28 Thread Matus Honek
Hi,

functionality seems to work fine. I have not checked the code thoroughly.
Kind of a test is attached (requires setting named's ldap connection 
appropriately).

ACK

Matúš Honěk


- Original Message -
From: Petr Spacek pspa...@redhat.com
To: tho...@redhat.com, Matus Honek mho...@redhat.com
Cc: freeipa-devel@redhat.com
Sent: Wednesday, May 27, 2015 2:50:52 PM
Subject: [PATCH 0377-0382] Synchronize changes from LDAP after reconnect

Hello,

https://fedorahosted.org/bind-dyndb-ldap/ticket/128

Previously records deleted when connection to LDAP server was down were not
synchronized properly. It should work now.

I use this command to simulate broken connections and connection 
re-establishment:
$ socat tcp-listen:3899,reuseaddr,fork tcp-connect:localhost:389

It should be enough to add ldap://$(hostname):3899 as LDAP URI to
/etc/named.conf and then simulate changes by killing and restarting socat.

Let me know if you need any assistance!

-- 
Petr^2 Spacek


test-resync-on-unavailability.sh
Description: application/shellscript
-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

[Freeipa-devel] [PATCH 0377-0382] Synchronize changes from LDAP after reconnect

2015-05-27 Thread Petr Spacek
Hello,

https://fedorahosted.org/bind-dyndb-ldap/ticket/128

Previously records deleted when connection to LDAP server was down were not
synchronized properly. It should work now.

I use this command to simulate broken connections and connection 
re-establishment:
$ socat tcp-listen:3899,reuseaddr,fork tcp-connect:localhost:389

It should be enough to add ldap://$(hostname):3899 as LDAP URI to
/etc/named.conf and then simulate changes by killing and restarting socat.

Let me know if you need any assistance!

-- 
Petr^2 Spacek
From 57e87e325bbfe60709a53c8d5422339bb5f2b664 Mon Sep 17 00:00:00 2001
From: Petr Spacek pspa...@redhat.com
Date: Tue, 26 May 2015 09:32:59 +0200
Subject: [PATCH] Add functions for MetaLDAP generation number manipulation.

https://fedorahosted.org/bind-dyndb-ldap/ticket/128
---
 src/mldap.c | 38 --
 src/mldap.h |  6 ++
 2 files changed, 42 insertions(+), 2 deletions(-)

diff --git a/src/mldap.c b/src/mldap.c
index 28fd2296b7ba6b97d1688ccf0be8ae63bbf6c2e8..d207df52ed92975c5d678add6dadfbcedfa052b8 100644
--- a/src/mldap.c
+++ b/src/mldap.c
@@ -18,6 +18,7 @@
 #include dns/enumclass.h
 #include dns/name.h
 #include dns/types.h
+#include dns/update.h
 
 #include ldap_entry.h
 #include metadb.h
@@ -60,7 +61,7 @@ mldap_new(isc_mem_t *mctx, mldapdb_t **mldapp) {
 	isc_mem_attach(mctx, mldap-mctx);
 
 	CHECK(metadb_new(mctx, mldap-mdb));
-	mldap-generation = 0;
+	mldap-generation = 1;
 
 	*mldapp = mldap;
 	return result;
@@ -99,6 +100,34 @@ mldap_closeversion(mldapdb_t *mldap, isc_boolean_t commit) {
 }
 
 /**
+ * Atomically increment MetaLDAP generation number.
+ */
+void mldap_cur_generation_bump(mldapdb_t *mldap) {
+	isc_uint32_t oldgen;
+	isc_uint32_t curgen;
+	isc_uint32_t newgen;
+
+	REQUIRE(mldap != NULL);
+
+	curgen = isc_atomic_cmpxchg((isc_int32_t *)mldap-generation, 0, 0);
+	do {
+		oldgen = curgen;
+		newgen = dns_update_soaserial(oldgen, dns_updatemethod_increment);
+		curgen = isc_atomic_cmpxchg((isc_int32_t *)mldap-generation, oldgen, newgen);
+	} while (curgen != oldgen);
+}
+
+/**
+ * Get current MetaLDAP generation number.
+ *
+ * Generation numbers have to be compared using isc_serial_* functions.
+ */
+isc_uint32_t
+mldap_cur_generation_get(mldapdb_t *mldap) {
+	return isc_atomic_cmpxchg((isc_int32_t *)mldap-generation, 0, 0);
+}
+
+/**
  * Convert UUID to 01234567-89ab-cdef-0123-456789abcdef.uuid.ldap. DNS name.
  *
  * @param[in]  beruuid
@@ -191,12 +220,17 @@ mldap_generation_store(mldapdb_t *mldap, metadb_node_t *node) {
 	unsigned char buff[sizeof(mldap-generation)];
 	isc_region_t region = { .base = buff, .length = sizeof(buff) };
 	dns_rdata_t rdata;
+	isc_uint32_t generation;
+
+	STATIC_ASSERT((sizeof(((mldapdb_t *)0)-generation) == sizeof(generation)), \
+		   mldapdb_t-generation and local generation size does not match);
 
 	dns_rdata_init(rdata);
 
 	/* Bytes should be in network-order but we do not care because:
 	 * 1) It is used only internally and always compared on this machine. */
-	memcpy(buff, mldap-generation, sizeof(mldap-generation));
+	generation = mldap_cur_generation_get(mldap);
+	memcpy(buff, generation, sizeof(generation));
 	dns_rdata_fromregion(rdata, dns_rdataclass_in, dns_rdatatype_a, region);
 	CHECK(metadb_rdata_store(rdata, node));
 
diff --git a/src/mldap.h b/src/mldap.h
index 752ab28b2da956817ad27c9c928863bc9e601c0c..c7eb725d2c1743da6037a2dc95d6bc000ccb0928 100644
--- a/src/mldap.h
+++ b/src/mldap.h
@@ -42,4 +42,10 @@ mldap_dnsname_get(metadb_node_t *node, dns_name_t *fqdn, dns_name_t *zone);
 isc_result_t ATTR_CHECKRESULT ATTR_NONNULLS
 mldap_dnsname_store(dns_name_t *fqdn, dns_name_t *zone, metadb_node_t *node);
 
+void ATTR_NONNULLS
+mldap_cur_generation_bump(mldapdb_t *mldap);
+
+isc_uint32_t ATTR_CHECKRESULT ATTR_NONNULLS
+mldap_cur_generation_get(mldapdb_t *mldap);
+
 #endif /* SRC_MLDAP_H_ */
-- 
2.1.0

From b476041bd6a88b88cd1739e61960a666868e1b23 Mon Sep 17 00:00:00 2001
From: Petr Spacek pspa...@redhat.com
Date: Tue, 26 May 2015 09:34:58 +0200
Subject: [PATCH] Increment MetaLDAP generation number on reconnect.

https://fedorahosted.org/bind-dyndb-ldap/ticket/128
---
 src/ldap_helper.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/src/ldap_helper.c b/src/ldap_helper.c
index abda9f54e84f097214ac0ba69a5cd657a39b5862..e9d2229d5369fc9ac9d692f426a2bdab99958184 100644
--- a/src/ldap_helper.c
+++ b/src/ldap_helper.c
@@ -4588,6 +4588,7 @@ ldap_syncrepl_watcher(isc_threadarg_t arg)
 			sane_sleep(inst, 1);
 			continue;
 		}
+		mldap_cur_generation_bump(inst-mldapdb);
 
 		log_info(LDAP instance '%s' is being synchronized, 
 			 please ignore message 'all zones loaded',
-- 
2.1.0

From 77ecee87f551567b94bd26290c734c7feb5ed93f Mon Sep 17 00:00:00 2001
From: Petr Spacek pspa...@redhat.com
Date: Wed, 27 May 2015 08:02:57 +0200
Subject: [PATCH] Add iterators for dead nodes in metaLDAP.

https://fedorahosted.org/bind-dyndb-ldap/ticket/128
---
 src/metadb.c |  55