Re: [SCM] CTDB repository - branch 1.2.40 updated - ctdb-1.2.70-2-gf0416af

2013-12-08 Thread ronnie sahlberg
Are you really bouncing the lock manager still?
You need to make sure that they enter a grace period across the
cluster so you can have lock revocery working semi-reliably.




On Sun, Dec 8, 2013 at 6:46 PM, Amitay Isaacs ami...@samba.org wrote:
 The branch, 1.2.40 has been updated
via  f0416afa6dc776489e2c7b3f0ede3480a3035bdb (commit)
via  bcb73f0b9c48fab6ac89e4ec0a297fd156beb6f6 (commit)
   from  5620a403f3ed8dd1ec8dcb449de4cf8a840893a8 (commit)

 http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=1.2.40


 - Log -
 commit f0416afa6dc776489e2c7b3f0ede3480a3035bdb
 Author: Martin Schwenke mar...@meltin.net
 Date:   Mon Dec 9 12:02:47 2013 +1100

 New version 1.2.71

 Signed-off-by: Martin Schwenke mar...@meltin.net

 commit bcb73f0b9c48fab6ac89e4ec0a297fd156beb6f6
 Author: Martin Schwenke mar...@meltin.net
 Date:   Mon Dec 9 11:47:01 2013 +1100

 eventscripts: Do not restart NFS on reconfigure, just do statd notify

 See also commit 2629de72e1f37b5e46772c2ef8d8d0012fc4ed37 in the master
 branch.

 In this branch the NFS restart was accidentally re-added in
 30e05c8b14413f3e1150b01e856d4b51c675ea5c and formalised in
 3fd4da153703bc83cf102ad367dd20d873e938a0 when the mistake wasn't
 realised.

 Signed-off-by: Martin Schwenke mar...@meltin.net

 ---

 Summary of changes:
  config/events.d/60.nfs |   17 ++---
  packaging/RPM/ctdb.spec.in |4 +++-
  2 files changed, 13 insertions(+), 8 deletions(-)


 Changeset truncated at 500 lines:

 diff --git a/config/events.d/60.nfs b/config/events.d/60.nfs
 index f567c82..13a2b04 100755
 --- a/config/events.d/60.nfs
 +++ b/config/events.d/60.nfs
 @@ -14,7 +14,16 @@ start_nfs() {
  service_name=nfs
  service_start=start_nfs
  service_stop=startstop_nfs stop
 -service_reconfigure=startstop_nfs restart
 +service_reconfigure=reconfigure_nfs
 +
 +reconfigure_nfs ()
 +{
 +# If IPs have been reallocated, we must restart the lock manager
 +# across all nodes and notify statd clients.
 +if [ -x $CTDB_BASE/statd-callout ] ; then
 +   $CTDB_BASE/statd-callout notify 
 +fi /dev/null 21
 +}

  nfs_check_thread_count ()
  {
 @@ -200,12 +209,6 @@ case $1 in
 ctdb_service_reconfigure
 exit 0
 fi
 -
 -   # if the ips have been reallocated, we must restart the lockmanager
 -   # across all nodes and ping all statd listeners
 -   [ -x $CTDB_BASE/statd-callout ]  {
 -   $CTDB_BASE/statd-callout notify 
 -   } /dev/null 21
 ;;
  *)
 ctdb_standard_event_handler $@
 diff --git a/packaging/RPM/ctdb.spec.in b/packaging/RPM/ctdb.spec.in
 index 2792b59..715a824 100644
 --- a/packaging/RPM/ctdb.spec.in
 +++ b/packaging/RPM/ctdb.spec.in
 @@ -3,7 +3,7 @@ Name: ctdb
  Summary: Clustered TDB
  Vendor: Samba Team
  Packager: Samba Team sa...@samba.org
 -Version: 1.2.70
 +Version: 1.2.71
  Release: 1GITHASH
  Epoch: 0
  License: GNU GPL version 3
 @@ -155,6 +155,8 @@ development libraries for ctdb

  %changelog

 +* Mon Dec 09 2013 : Version 1.2.71
 +  - Don't restart NFS on failover, just do statd notification
  * Mon Nov 25 2013 : Version 1.2.70
- Just warn when recovery fails to update flags on inactive node
  * Mon Nov 04 2013 : Version 1.2.69


 --
 CTDB repository


[SCM] CTDB repository - branch 1.2.40 updated - ctdb-1.9.1-570-g8c3aed3

2012-06-28 Thread Ronnie Sahlberg
The branch, 1.2.40 has been updated
   via  8c3aed36615e083e0b91efd70380b7711f9f9f7e (commit)
   via  0aa9a450ef8919acf9c6e3309712ed31fc7e70cc (commit)
   via  9d8a6d792c40eed02f01a657c4fd252e9aae7039 (commit)
   via  bf0ce09db2302ef52ea0861cb8079c8b7220ec79 (commit)
   via  ee3cbeb874cf0dd1d91ddf3ebb9e4bd94fb15169 (commit)
   via  183650afc505f5a3025ccbf7c6e07778761223c7 (commit)
   via  ce82656a88ec6cd3955b071f87bfa611abe0a94a (commit)
   via  c016f65ff147f3e92a39a26fcc086ca7992b1663 (commit)
   via  47197d7571a20d047b6cf19597e8c9d7c3064f24 (commit)
   via  7e6a846aca704aa5394631e1fe35d893b881b8a6 (commit)
   via  b00fcf9af9a2e2dad0b2ca282ab03b7b70a1493b (commit)
  from  b54938c8dfaac046aee9eec92724e7ba2153d65a (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=1.2.40


- Log -
commit 8c3aed36615e083e0b91efd70380b7711f9f9f7e
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Fri Jun 29 12:31:13 2012 +1000

New version 1.2.44

commit 0aa9a450ef8919acf9c6e3309712ed31fc7e70cc
Author: Martin Schwenke mar...@meltin.net
Date:   Thu Jun 21 14:18:35 2012 +1000

eventscripts: 13.per_ip_routing - flock should have a timeout

... and flock failure should be fatal.

Signed-off-by: Martin Schwenke mar...@meltin.net

commit 9d8a6d792c40eed02f01a657c4fd252e9aae7039
Author: Martin Schwenke mar...@meltin.net
Date:   Thu Jun 21 14:17:06 2012 +1000

eventscripts: 13.per_ip_routing - readability improvement

Signed-off-by: Martin Schwenke mar...@meltin.net

commit bf0ce09db2302ef52ea0861cb8079c8b7220ec79
Author: Martin Schwenke mar...@meltin.net
Date:   Fri Jun 15 17:22:02 2012 +1000

eventscripts: 13.per_ip_routing should remove bogus routes on ipreallocated

Signed-off-by: Martin Schwenke mar...@meltin.net

commit ee3cbeb874cf0dd1d91ddf3ebb9e4bd94fb15169
Author: Martin Schwenke mar...@meltin.net
Date:   Wed Jun 13 13:49:49 2012 +1000

eventscripts: Print a warning on failure to delete a routing rule

del_routing_for_ip() currently fails silently, which could hide real
errors.

In add_routing_for_ip() we don't want to see any error when calling
del_routing_for_ip(), since we don't expect the rule to be there.

Signed-off-by: Martin Schwenke mar...@meltin.net

commit 183650afc505f5a3025ccbf7c6e07778761223c7
Author: Martin Schwenke mar...@meltin.net
Date:   Tue Apr 24 17:17:07 2012 +1000

Eventscripts: fix basename - dirname typo

I fixed one of these previously but didn't notice this one...  :-(

Signed-off-by: Martin Schwenke mar...@meltin.net

commit ce82656a88ec6cd3955b071f87bfa611abe0a94a
Author: Martin Schwenke mar...@meltin.net
Date:   Wed Apr 18 16:49:48 2012 +1000

Eventscripts - Fix typo in 13.per_ip_routing support for __auto_link_local__

Signed-off-by: Martin Schwenke mar...@meltin.net

commit c016f65ff147f3e92a39a26fcc086ca7992b1663
Author: Martin Schwenke mar...@meltin.net
Date:   Tue Mar 20 16:23:24 2012 +1100

Eventscripts - make 13.per_ip_routing fail gracefully if config is missing

Currently it spews out random messages about the file being missing.

Signed-off-by: Martin Schwenke mar...@meltin.net

commit 47197d7571a20d047b6cf19597e8c9d7c3064f24
Author: Martin Schwenke mar...@meltin.net
Date:   Tue Mar 20 16:21:09 2012 +1100

Eventscripts - make 13.per_ip_routing try harder to find public_addresses

Signed-off-by: Martin Schwenke mar...@meltin.net

commit 7e6a846aca704aa5394631e1fe35d893b881b8a6
Author: Martin Schwenke mar...@meltin.net
Date:   Tue Mar 20 16:20:10 2012 +1100

Eventscripts - use set_proc() rather than accessing /proc directly

Signed-off-by: Martin Schwenke mar...@meltin.net

commit b00fcf9af9a2e2dad0b2ca282ab03b7b70a1493b
Author: Martin Schwenke mar...@meltin.net
Date:   Tue Mar 20 16:18:49 2012 +1100

Eventscripts - 13.per_ip_routing should use dirname not basename for mkdir

Signed-off-by: Martin Schwenke mar...@meltin.net

---

Summary of changes:
 config/events.d/13.per_ip_routing |   75 +
 packaging/RPM/ctdb.spec.in|4 +-
 2 files changed, 62 insertions(+), 17 deletions(-)


Changeset truncated at 500 lines:

diff --git a/config/events.d/13.per_ip_routing 
b/config/events.d/13.per_ip_routing
index d17342d..f16a74e 100755
--- a/config/events.d/13.per_ip_routing
+++ b/config/events.d/13.per_ip_routing
@@ -93,7 +93,7 @@ ensure_table_id_for_ip ()
 _f=$CTDB_ETCDIR/iproute2/rt_tables
 # This file should always exist, but...
 if [ ! -f $_f ] ; then
-   mkdir -p $(basename $_f)
+   mkdir -p $(dirname $_f)
touch $_f
 fi
 
@@ -105,7 +105,9 @@ ensure_table_id_for_ip ()
 # new unused one (that is greater than all the used ones in the
 # range

[SCM] CTDB repository - branch 1.13 updated - ctdb-1.43-16-g663e425

2012-06-19 Thread Ronnie Sahlberg
The branch, 1.13 has been updated
   via  663e425084d3961ad6839b2649eeb981d6e36502 (commit)
   via  3bdf5523a031ea7d74264ebef2d1e1eecc772ded (commit)
   via  da5e11dbe33662bc14126787da09a9ef39809fc7 (commit)
  from  0cd522f854bb788317e15e5f9a562bdb5abcfb17 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=1.13


- Log -
commit 663e425084d3961ad6839b2649eeb981d6e36502
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Wed Jun 20 15:10:05 2012 +1000

When we find an ip we shouldnt host, just release it

Dont call a full blown clusterwide ipreallocation,  just release it locally

commit 3bdf5523a031ea7d74264ebef2d1e1eecc772ded
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Wed Jun 20 10:08:11 2012 +1000

When we release an ip, get the interface name from the kernel

instead of using the interface where ctdb thinks the ip is hosted at.
The difference is that this now allows us to handle cases where we want to 
release an ip   but ctdbd does not know which interface the ip is assigned on.
(user has used 'ip addr add...'  and manually assigned an ip to the wrong 
interface)

commit da5e11dbe33662bc14126787da09a9ef39809fc7
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Wed Jun 20 13:32:02 2012 +1000

Add new command to find which interface is located on

---

Summary of changes:
 common/system_common.c |   82 
 include/ctdb_private.h |1 +
 server/ctdb_recoverd.c |8 +++-
 server/ctdb_takeover.c |   22 +---
 tools/ctdb.c   |   22 +
 5 files changed, 120 insertions(+), 15 deletions(-)


Changeset truncated at 500 lines:

diff --git a/common/system_common.c b/common/system_common.c
index 3fe90e6..6ee615f 100644
--- a/common/system_common.c
+++ b/common/system_common.c
@@ -75,3 +75,85 @@ bool ctdb_sys_have_ip(ctdb_sock_addr *_addr)
 }
 
 
+/* find which interface an ip address is currently assigned to */
+char *ctdb_sys_find_ifname(ctdb_sock_addr *addr)
+{
+   int s;
+   int size;
+   struct ifconf ifc;
+   char *ptr;
+
+   s = socket(AF_INET, SOCK_RAW, htons(IPPROTO_RAW));
+   if (s == -1) {
+   DEBUG(DEBUG_CRIT,(__location__  failed to open raw socket 
(%s)\n,
+strerror(errno)));
+   return NULL;
+   }
+
+
+   size = sizeof(struct ifreq);
+   ifc.ifc_buf = NULL;
+   ifc.ifc_len = size;
+
+   while(ifc.ifc_len  (size - sizeof(struct ifreq))) {
+   size *= 2;
+
+   free(ifc.ifc_buf);  
+   ifc.ifc_len = size;
+   ifc.ifc_buf = malloc(size);
+   memset(ifc.ifc_buf, 0, size);
+   if (ioctl(s, SIOCGIFCONF, (caddr_t)ifc)  0) {
+   DEBUG(DEBUG_CRIT,(Failed to read ifc buffer from 
socket\n));
+   free(ifc.ifc_buf);  
+   close(s);
+   return NULL;
+   }
+   }
+
+   for (ptr =(char *)ifc.ifc_buf; ptr  ((char *)ifc.ifc_buf) + 
ifc.ifc_len; ) {
+   char *ifname;
+   struct ifreq *ifr;
+
+   ifr = (struct ifreq *)ptr;
+
+#ifdef HAVE_SOCKADDR_LEN
+   if (ifr-ifr_addr.sa_len  sizeof(struct sockaddr)) {
+   ptr += sizeof(ifr-ifr_name) + ifr-ifr_addr.sa_len;
+   } else {
+   ptr += sizeof(ifr-ifr_name) + sizeof(struct sockaddr);
+   }
+#else
+   ptr += sizeof(struct ifreq);
+#endif
+
+   if (ifr-ifr_addr.sa_family != addr-sa.sa_family) {
+   continue;
+   }
+
+   switch (addr-sa.sa_family) {
+   case AF_INET:
+
+
+   if (memcmp(addr-ip.sin_addr, ((struct sockaddr_in 
*)ifr-ifr_addr)-sin_addr, sizeof(addr-ip.sin_addr))) {
+   continue;
+   }
+   break;
+   case AF_INET6:
+   if (memcmp(addr-ip6.sin6_addr, ((struct sockaddr_in6 
*)ifr-ifr_addr)-sin6_addr, sizeof(addr-ip6.sin6_addr))) {
+   continue;
+   }
+   break;
+   }
+
+   ifname = strdup(ifr-ifr_name);
+   free(ifc.ifc_buf);  
+   close(s);
+   return ifname;
+   }
+
+
+   free(ifc.ifc_buf);  
+   close(s);
+
+   return NULL;
+}
diff --git a/include/ctdb_private.h b/include/ctdb_private.h
index 7c3fdf0..0f9da36 100644
--- a/include/ctdb_private.h
+++ b/include/ctdb_private.h
@@ -1144,6 +1144,7 @@ int ctdb_ctrl_set_iface_link(struct ctdb_context *ctdb,
 uint32_t uint16_checksum(uint16_t *data, size_t n);
 int ctdb_sys_send_arp(const ctdb_sock_addr *addr

[SCM] CTDB repository - branch master updated - ctdb-1.13-205-g9a806de

2012-06-19 Thread Ronnie Sahlberg
The branch, master has been updated
   via  9a806dec8687e2ec08a308853b61af6aed5e5d1e (commit)
   via  c6bf22ba5c01001b7febed73dd16a03bd3fd2bed (commit)
   via  f07376309e70f5ccdb7de8453caacc71b451ab48 (commit)
  from  8307c70ed98996b430c470e9641a09fdeeb81bd8 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=master


- Log -
commit 9a806dec8687e2ec08a308853b61af6aed5e5d1e
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Wed Jun 20 15:10:05 2012 +1000

When we find an ip we shouldnt host, just release it

Dont call a full blown clusterwide ipreallocation,  just release it locally

commit c6bf22ba5c01001b7febed73dd16a03bd3fd2bed
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Wed Jun 20 10:08:11 2012 +1000

When we release an ip, get the interface name from the kernel

instead of using the interface where ctdb thinks the ip is hosted at.
The difference is that this now allows us to handle cases where we want to 
release an ip   but ctdbd does not know which interface the ip is assigned on.
(user has used 'ip addr add...'  and manually assigned an ip to the wrong 
interface)

commit f07376309e70f5ccdb7de8453caacc71b451ab48
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Wed Jun 20 13:32:02 2012 +1000

Add new command to find which interface is located on

---

Summary of changes:
 common/system_common.c |   82 
 include/ctdb_private.h |1 +
 server/ctdb_recoverd.c |8 +++-
 server/ctdb_takeover.c |   22 +---
 tools/ctdb.c   |   22 +
 5 files changed, 120 insertions(+), 15 deletions(-)


Changeset truncated at 500 lines:

diff --git a/common/system_common.c b/common/system_common.c
index 3fe90e6..6ee615f 100644
--- a/common/system_common.c
+++ b/common/system_common.c
@@ -75,3 +75,85 @@ bool ctdb_sys_have_ip(ctdb_sock_addr *_addr)
 }
 
 
+/* find which interface an ip address is currently assigned to */
+char *ctdb_sys_find_ifname(ctdb_sock_addr *addr)
+{
+   int s;
+   int size;
+   struct ifconf ifc;
+   char *ptr;
+
+   s = socket(AF_INET, SOCK_RAW, htons(IPPROTO_RAW));
+   if (s == -1) {
+   DEBUG(DEBUG_CRIT,(__location__  failed to open raw socket 
(%s)\n,
+strerror(errno)));
+   return NULL;
+   }
+
+
+   size = sizeof(struct ifreq);
+   ifc.ifc_buf = NULL;
+   ifc.ifc_len = size;
+
+   while(ifc.ifc_len  (size - sizeof(struct ifreq))) {
+   size *= 2;
+
+   free(ifc.ifc_buf);  
+   ifc.ifc_len = size;
+   ifc.ifc_buf = malloc(size);
+   memset(ifc.ifc_buf, 0, size);
+   if (ioctl(s, SIOCGIFCONF, (caddr_t)ifc)  0) {
+   DEBUG(DEBUG_CRIT,(Failed to read ifc buffer from 
socket\n));
+   free(ifc.ifc_buf);  
+   close(s);
+   return NULL;
+   }
+   }
+
+   for (ptr =(char *)ifc.ifc_buf; ptr  ((char *)ifc.ifc_buf) + 
ifc.ifc_len; ) {
+   char *ifname;
+   struct ifreq *ifr;
+
+   ifr = (struct ifreq *)ptr;
+
+#ifdef HAVE_SOCKADDR_LEN
+   if (ifr-ifr_addr.sa_len  sizeof(struct sockaddr)) {
+   ptr += sizeof(ifr-ifr_name) + ifr-ifr_addr.sa_len;
+   } else {
+   ptr += sizeof(ifr-ifr_name) + sizeof(struct sockaddr);
+   }
+#else
+   ptr += sizeof(struct ifreq);
+#endif
+
+   if (ifr-ifr_addr.sa_family != addr-sa.sa_family) {
+   continue;
+   }
+
+   switch (addr-sa.sa_family) {
+   case AF_INET:
+
+
+   if (memcmp(addr-ip.sin_addr, ((struct sockaddr_in 
*)ifr-ifr_addr)-sin_addr, sizeof(addr-ip.sin_addr))) {
+   continue;
+   }
+   break;
+   case AF_INET6:
+   if (memcmp(addr-ip6.sin6_addr, ((struct sockaddr_in6 
*)ifr-ifr_addr)-sin6_addr, sizeof(addr-ip6.sin6_addr))) {
+   continue;
+   }
+   break;
+   }
+
+   ifname = strdup(ifr-ifr_name);
+   free(ifc.ifc_buf);  
+   close(s);
+   return ifname;
+   }
+
+
+   free(ifc.ifc_buf);  
+   close(s);
+
+   return NULL;
+}
diff --git a/include/ctdb_private.h b/include/ctdb_private.h
index 7c3fdf0..0f9da36 100644
--- a/include/ctdb_private.h
+++ b/include/ctdb_private.h
@@ -1144,6 +1144,7 @@ int ctdb_ctrl_set_iface_link(struct ctdb_context *ctdb,
 uint32_t uint16_checksum(uint16_t *data, size_t n);
 int ctdb_sys_send_arp(const ctdb_sock_addr *addr

[SCM] CTDB repository - branch 1.13 updated - ctdb-1.43-17-g7f252cb

2012-06-19 Thread Ronnie Sahlberg
The branch, 1.13 has been updated
   via  7f252cb7f3b12bdca960bbeecc5424adb34ce3be (commit)
  from  663e425084d3961ad6839b2649eeb981d6e36502 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=1.13


- Log -
commit 7f252cb7f3b12bdca960bbeecc5424adb34ce3be
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Wed Jun 20 15:26:12 2012 +1000

New version 1.44

---

Summary of changes:
 packaging/RPM/ctdb.spec.in |6 ++
 1 files changed, 6 insertions(+), 0 deletions(-)


Changeset truncated at 500 lines:

diff --git a/packaging/RPM/ctdb.spec.in b/packaging/RPM/ctdb.spec.in
index 33da7e5..2297487 100644
--- a/packaging/RPM/ctdb.spec.in
+++ b/packaging/RPM/ctdb.spec.in
@@ -199,6 +199,12 @@ test suite for ctdb
 
 
 %changelog
+* Wed Jun 20 2012 : Version 1.44
+ - Add back tracking of long running events
+ - Add checks and logigng when a database contains a very large number of 
records or very large records
+ - When performing database recovery, preallocate the marshalling buffer in 
large chunks instead of calling a sometimes very expensive reallo()
+ - test updates
+ - add new function to automatically find which interface an ip address is 
hosted on
 * Thu May 17 2012 : Version 1.32
  - Update Ganesha script
  - Add new script to collect data when eventscripts hang


-- 
CTDB repository


[SCM] CTDB repository - annotated tag ctdb-1.44 created - ctdb-1.44

2012-06-19 Thread Ronnie Sahlberg
The annotated tag, ctdb-1.44 has been created
at  75e5905698030de12d8ae45d1f01ec14acd80244 (tag)
   tagging  7f252cb7f3b12bdca960bbeecc5424adb34ce3be (commit)
  replaces  ctdb-1.43
 tagged by  Ronnie Sahlberg
on  Wed Jun 20 15:26:43 2012 +1000

- Log -
tag for 1.44
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)

iQEcBAABAgAGBQJP4V8cAAoJEB6eS+vwPaeGKuoH/jsLQ3KC1rcQsq/nrNRdIqi9
0pk8BGj/G+KqL/UPfIwfRQuql6TCINXfZfsEv2o48RAVYnbOz65j4BKul7uON305
oE1E/3WoxGbWX/pOz2yalpsXcRwDYi6p3M52wQZ/VgPyTERN3ZFa46uKcwpIhWGU
IMEv9Llhgy1XZip9jd6DMfQkR8F69hPISyBNlSBhtI0SitK+Cz9iCmelYm8Kzlmq
+gKc8Fv1PofHhAtzIeBqzd+Vd4+gFbGZBvRtuPkd+tB+Hm9cxuTjx/cioEpyT/Vb
aeK4nbLZMDMPhGmGJHW7Ncba2gRVCuZ9i9jRLosuJDXMicBua/2ksAX2Uu/XAHc=
=2CeJ
-END PGP SIGNATURE-

Amitay Isaacs (2):
  tests: Use per node log files when running tests with local daemons
  packaging: make ctdb-tests package depend on nc

Martin Schwenke (4):
  tests: test_wrap needs to set TEST_BIN_DIR when installed
  Eventscripts: Fix deprecated iptables ! usage
  tests: Complex tests must not be run from a cluster node
  tests/eventscripts: Tweak expected output for lockd:b restart

Ronnie Sahlberg (11):
  TEVENT: Add back tracking of long runnig  events to the local copy of 
tevent library
  DEBUG: Add checks for and print debug messages when 1) a database 
contains very many records, 2) when a database is very big, 3) when a single 
record is very big.
  DOCS: Document the new tunables to produce warnings if databases grow 
unexpectedly big.
  RECOVER: When we pull databases during recovery, we used to reallocate 
the databuffer for each entry added. This would normally not be an issue, but 
for cases where memory is fragmented, this could start to cost significant cpu 
if we need to reallocate and move to a different region.
  RECOVERY: Increase the time we allow before timing out recovery related 
tasks.
  Run the shutdown eventscript before we tear down the transport
  STATISTICS: Add tracking of the 10 hottest keys per database measured in 
hopcount
  Add new command to find which interface is located on
  When we release an ip, get the interface name from the kernel
  When we find an ip we shouldnt host, just release it
  New version 1.44

---


-- 
CTDB repository


[SCM] CTDB repository - branch 1.13 updated - ctdb-1.43-13-g0cd522f

2012-06-13 Thread Ronnie Sahlberg
The branch, 1.13 has been updated
   via  0cd522f854bb788317e15e5f9a562bdb5abcfb17 (commit)
  from  ba94a0a9ccad7c1de0939e74f0163ae41102 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=1.13


- Log -
commit 0cd522f854bb788317e15e5f9a562bdb5abcfb17
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Wed Jun 13 16:17:18 2012 +1000

STATISTICS: Add tracking of the 10 hottest keys per database measured in 
hopcount

and add mechanisms to dump it using the ctdb dbstatistics command

---

Summary of changes:
 include/ctdb_protocol.h   |   14 
 libctdb/control.c |   40 ---
 server/ctdb_call.c|   50 -
 server/ctdb_ltdb_server.c |   38 -
 tools/ctdb.c  |9 
 5 files changed, 144 insertions(+), 7 deletions(-)


Changeset truncated at 500 lines:

diff --git a/include/ctdb_protocol.h b/include/ctdb_protocol.h
index 5c787ff..33187c7 100644
--- a/include/ctdb_protocol.h
+++ b/include/ctdb_protocol.h
@@ -614,6 +614,7 @@ struct ctdb_traverse_start_ext {
   ctdb statistics information
  */
 #define MAX_COUNT_BUCKETS 16
+#define MAX_HOT_KEYS  10
 
 struct ctdb_statistics {
uint32_t num_clients;
@@ -680,10 +681,23 @@ struct ctdb_statistics_wire {
 /*
  * db statistics
  */
+struct ctdb_db_hot_key {
+   uint32_t count;
+   TDB_DATA key;
+};
 struct ctdb_db_statistics {
uint32_t db_ro_delegations;
uint32_t db_ro_revokes;
uint32_t hop_count_bucket[MAX_COUNT_BUCKETS];
+   uint32_t num_hot_keys;
+   struct ctdb_db_hot_key hot_keys[MAX_HOT_KEYS];
+};
+struct ctdb_db_statistics_wire {
+   uint32_t db_ro_delegations;
+   uint32_t db_ro_revokes;
+   uint32_t hop_count_bucket[MAX_COUNT_BUCKETS];
+   uint32_t num_hot_keys;
+   char hot_keys[1];
 };
 
 /*
diff --git a/libctdb/control.c b/libctdb/control.c
index b4c54cd..f927e08 100644
--- a/libctdb/control.c
+++ b/libctdb/control.c
@@ -120,6 +120,9 @@ bool ctdb_getdbstat_recv(struct ctdb_connection *ctdb,
 {
struct ctdb_reply_control *reply;
struct ctdb_db_statistics *s;
+   struct ctdb_db_statistics_wire *wire;
+   int i;
+   char *ptr;
 
reply = unpack_reply_control(req, CTDB_CONTROL_GET_DB_STATISTICS);
if (!reply) {
@@ -129,16 +132,36 @@ bool ctdb_getdbstat_recv(struct ctdb_connection *ctdb,
DEBUG(ctdb, LOG_ERR, ctdb_getpnn_recv: status -1);
return false;
}
-   if (reply-datalen != sizeof(struct ctdb_db_statistics)) {
-   DEBUG(ctdb, LOG_ERR, ctdb_getdbstat_recv: returned data is %d 
bytes but should be %d, reply-datalen, (int)sizeof(struct 
ctdb_db_statistics));
+   if (reply-datalen  offsetof(struct ctdb_db_statistics_wire, 
hot_keys)) {
+   DEBUG(ctdb, LOG_ERR, ctdb_getdbstat_recv: returned data is %d 
bytes but should be = %d, reply-datalen, (int)sizeof(struct 
ctdb_db_statistics));
return false;
}
 
-   s = malloc(sizeof(struct ctdb_db_statistics));
+   wire = reply-data;
+
+   s = malloc(offsetof(struct ctdb_db_statistics, hot_keys) + 
sizeof(struct ctdb_db_hot_key) * wire-num_hot_keys);
if (!s) {
return false;
}
-   memcpy(s, reply-data, sizeof(struct ctdb_db_statistics));
+   s-db_ro_delegations = wire-db_ro_delegations;
+   s-db_ro_revokes = wire-db_ro_revokes;
+   for (i = 0; i  MAX_COUNT_BUCKETS; i++) {
+   s-hop_count_bucket[i] = wire-hop_count_bucket[i];
+   }
+   s-num_hot_keys  = wire-num_hot_keys;
+   ptr = wire-hot_keys[0];
+   for (i = 0; i  wire-num_hot_keys; i++) {
+   s-hot_keys[i].count = *(uint32_t *)ptr;
+   ptr += 4;
+
+   s-hot_keys[i].key.dsize = *(uint32_t *)ptr;
+   ptr += 4;
+
+   s-hot_keys[i].key.dptr = malloc(s-hot_keys[i].key.dsize);
+   memcpy(s-hot_keys[i].key.dptr, ptr, s-hot_keys[i].key.dsize);
+   ptr += s-hot_keys[i].key.dsize;
+   }
+
*stat = s;
 
return true;
@@ -158,9 +181,18 @@ struct ctdb_request *ctdb_getdbstat_send(struct 
ctdb_connection *ctdb,
 
 void ctdb_free_dbstat(struct ctdb_db_statistics *stat)
 {
+   int i;
+
if (stat == NULL) {
return;
}
+
+   for (i = 0; i  stat-num_hot_keys; i++) {
+   if (stat-hot_keys[i].key.dptr != NULL) {
+   free(stat-hot_keys[i].key.dptr);
+   }
+   }
+
free(stat);
 }
 
diff --git a/server/ctdb_call.c b/server/ctdb_call.c
index fe7e947..56cb5e8 100644
--- a/server/ctdb_call.c
+++ b/server/ctdb_call.c
@@ -667,6 +667,54 @@ ctdb_defer_pinned_down_request(struct ctdb_context *ctdb

[SCM] CTDB repository - branch master updated - ctdb-1.13-202-g8307c70

2012-06-13 Thread Ronnie Sahlberg
The branch, master has been updated
   via  8307c70ed98996b430c470e9641a09fdeeb81bd8 (commit)
   via  98e1b46adba11b9549b5c5976e1f561fe732fa6e (commit)
   via  0dc204988eadff214dd149a756d756ab6e96e410 (commit)
   via  7ebc00dc6a89043a971a720e7c21baf5f2a0233d (commit)
   via  cb2bbe93628c1ab932c2e1ad6e2ec199a98f74c6 (commit)
   via  88040778aace229d724de1ba7556aded12e22f86 (commit)
   via  e0c9200c05b1f7a04e002f505ebb5ba9340c0ca1 (commit)
   via  6559106b8b853920f325f2dba532f4008e931fa3 (commit)
  from  1a6a011c772f7d302d114d7c8a151fa7820ec85f (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=master


- Log -
commit 8307c70ed98996b430c470e9641a09fdeeb81bd8
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Wed Jun 13 16:17:18 2012 +1000

STATISTICS: Add tracking of the 10 hottest keys per database measured in 
hopcount

and add mechanisms to dump it using the ctdb dbstatistics command

commit 98e1b46adba11b9549b5c5976e1f561fe732fa6e
Author: Martin Schwenke mar...@meltin.net
Date:   Thu Jun 7 15:08:15 2012 +1000

Reimplement logging of long running events

Reimplement 5aba53e6adcfcd7edbdac9e30aa5fcba176aca00 using tevent
trace points.

Signed-off-by: Martin Schwenke mar...@meltin.net

commit 0dc204988eadff214dd149a756d756ab6e96e410
Author: Stefan Metzmacher me...@samba.org
Date:   Fri Jun 8 12:50:21 2012 +0200

tevent: change version to 0.9.16

This adds tevent_*_trace_*() and tevent_context_init_ops()

metze

Autobuild-User(master): Stefan Metzmacher me...@samba.org
Autobuild-Date(master): Fri Jun  8 20:47:41 CEST 2012 on sn-devel-104

commit 7ebc00dc6a89043a971a720e7c21baf5f2a0233d
Author: Stefan Metzmacher me...@samba.org
Date:   Fri May 11 15:19:55 2012 +0200

tevent: expose tevent_context_init_ops

This can be used to implement wrapper backends,
while passing a private pointer to the backens init function
via ev-additional_data.

metze

commit cb2bbe93628c1ab932c2e1ad6e2ec199a98f74c6
Author: Martin Schwenke mar...@meltin.net
Date:   Tue Jun 5 16:00:07 2012 +1000

lib/tevent: Add trace point callback

Set/get a single callback function to be invoked at various trace
points.  Define before wait and after wait trace points - more
trace points can be added later if required.

CTDB wants this to log long waits and events.

Pair-programmed-with: Amitay Isaacs ami...@gmail.com
Signed-off-by: Martin Schwenke mar...@meltin.net
Signed-off-by: Stefan Metzmacher me...@samba.org

commit 88040778aace229d724de1ba7556aded12e22f86
Author: Martin Schwenke mar...@meltin.net
Date:   Thu Jun 7 14:20:13 2012 +1000

Revert TEVENT: Add back tracking of long runnig  events to the local copy 
of tevent library

This reverts commit 5aba53e6adcfcd7edbdac9e30aa5fcba176aca00.

Do this using new tevent trace point callback.

commit e0c9200c05b1f7a04e002f505ebb5ba9340c0ca1
Author: Martin Schwenke mar...@meltin.net
Date:   Thu Jun 7 12:26:02 2012 +1000

lib/tevent: In poll_event_context, add a pointer back to the tevent_context

This makes it consistent with the other backends.

Signed-off-by: Martin Schwenke mar...@meltin.net
Signed-off-by: Stefan Metzmacher me...@samba.org

commit 6559106b8b853920f325f2dba532f4008e931fa3
Author: Stefan Metzmacher me...@samba.org
Date:   Mon May 14 11:48:00 2012 +0200

lib/tevent/testsuite: no longer use 'compat' symbols

metze

---

Summary of changes:
 include/ctdb_protocol.h|   14 +
 .../ABI/{tevent-0.9.15.sigs = tevent-0.9.16.sigs} |4 ++
 lib/tevent/testsuite.c |   24 
 lib/tevent/tevent.c|8 ++-
 lib/tevent/tevent.h|   57 
 lib/tevent/tevent_debug.c  |   24 
 lib/tevent/tevent_epoll.c  |6 +--
 lib/tevent/tevent_internal.h   |   10 +++-
 lib/tevent/tevent_poll.c   |6 ++
 lib/tevent/tevent_select.c |6 +--
 lib/tevent/tevent_standard.c   |6 +--
 lib/tevent/tevent_util.c   |   52 --
 libctdb/control.c  |   40 --
 server/ctdb_call.c |   50 +-
 server/ctdb_daemon.c   |   46 
 server/ctdb_ltdb_server.c  |   38 -
 tools/ctdb.c   |9 +++
 17 files changed, 312 insertions(+), 88 deletions(-)
 copy lib/tevent/ABI/{tevent-0.9.15.sigs = tevent-0.9.16.sigs} (94%)


Changeset

[SCM] CTDB repository - branch master updated - ctdb-1.13-193-gac89da4

2012-05-29 Thread Ronnie Sahlberg
The branch, master has been updated
   via  ac89da4eea98fa686408c5671a6c44c0fd1d7a58 (commit)
   via  0be452958db95c8253c362a1c08a1966e53a1f99 (commit)
   via  bc55e09fdac9f743d6428bfe0be77840ad0fd1ba (commit)
   via  6e5cbe8fff71985e5a2fc16b7e9f2b868011ff5d (commit)
  from  fd3b73d7e634f16cbb99d7d5a548e12f00d1aadb (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=master


- Log -
commit ac89da4eea98fa686408c5671a6c44c0fd1d7a58
Author: Amitay Isaacs ami...@gmail.com
Date:   Fri May 25 15:57:14 2012 +1000

tests: Increment RSN always in ctdb_update_record_persistent test

If the record does not exist in persistent DB, RSN for that record is
considered 0. To write a record, RSN for that record should be set to 1,
otherwise the RSN check would fail.

Signed-off-by: Amitay Isaacs ami...@gmail.com

commit 0be452958db95c8253c362a1c08a1966e53a1f99
Author: Amitay Isaacs ami...@gmail.com
Date:   Fri May 25 11:40:38 2012 +1000

tests: Fix ctdb_fetch test (parse extra lines of output)

Signed-off-by: Amitay Isaacs ami...@gmail.com

commit bc55e09fdac9f743d6428bfe0be77840ad0fd1ba
Author: Amitay Isaacs ami...@gmail.com
Date:   Thu May 24 16:46:07 2012 +1000

tests: Fix flakey behavior of ctdb_fetch test

There were two issues with this test:

1. Since the messages are sent from one node to the next, if a node
   does not register for messages before CTDB on that nodes receives
   the message, it will never be seen by ctdb_fetch and it would
   block on receive and would not send any messages to next node.
   The crude solution is to sleep just before the messages are sent,
   so that ctdb_fetch on all nodes have registered for the messages.

2. If ctdb_fetch stops sending messages after timelimit expiry, the
   next node will keep waiting to receive messages in event_loop_once().
   The default timeout is 30 seconds for event_loop_once(). Adding a
   timed event will always set the timeout value to the time remaining
   for the timed event to expire.

Signed-off-by: Amitay Isaacs ami...@gmail.com

commit 6e5cbe8fff71985e5a2fc16b7e9f2b868011ff5d
Author: Amitay Isaacs ami...@gmail.com
Date:   Thu May 17 16:08:37 2012 +1000

server: Replace BOOL datatype with bool, True/False with true/false

Signed-off-by: Amitay Isaacs ami...@gmail.com

---

Summary of changes:
 client/ctdb_client.c  |8 
 common/rb_tree.c  |6 +++---
 include/includes.h|7 +--
 lib/util/util.c   |2 +-
 server/ctdb_call.c|8 
 server/ctdb_control.c |2 +-
 server/ctdb_daemon.c  |4 ++--
 server/ctdb_freeze.c  |2 +-
 server/ctdb_takeover.c|2 +-
 server/ctdbd.c|2 +-
 tests/simple/52_ctdb_fetch.sh |2 +-
 tests/src/ctdb_fetch.c|   17 +
 tests/src/ctdb_update_record_persistent.c |4 ++--
 13 files changed, 39 insertions(+), 27 deletions(-)


Changeset truncated at 500 lines:

diff --git a/client/ctdb_client.c b/client/ctdb_client.c
index 69b0a04..9162a47 100644
--- a/client/ctdb_client.c
+++ b/client/ctdb_client.c
@@ -2023,7 +2023,7 @@ static void traverse_handler(struct ctdb_context *ctdb, 
uint64_t srvid, TDB_DATA
if (data.dsize  sizeof(uint32_t) ||
d-length != data.dsize) {
DEBUG(DEBUG_ERR,(Bad data size %u in traverse_handler\n, 
(unsigned)data.dsize));
-   state-done = True;
+   state-done = true;
return;
}
 
@@ -2034,7 +2034,7 @@ static void traverse_handler(struct ctdb_context *ctdb, 
uint64_t srvid, TDB_DATA
 
if (key.dsize == 0  data.dsize == 0) {
/* end of traverse */
-   state-done = True;
+   state-done = true;
return;
}
 
@@ -2046,7 +2046,7 @@ static void traverse_handler(struct ctdb_context *ctdb, 
uint64_t srvid, TDB_DATA
}
 
if (state-fn(ctdb, key, data, state-private_data) != 0) {
-   state-done = True;
+   state-done = true;
}
 
state-count++;
@@ -2071,7 +2071,7 @@ static int ctdb_traverse_ext(struct ctdb_db_context 
*ctdb_db,
uint64_t srvid = (getpid() | 0xFLL60);
struct traverse_state state;
 
-   state.done = False;
+   state.done = false;
state.count = 0;
state.private_data = private_data;
state.fn = fn;
diff --git a/common/rb_tree.c b/common/rb_tree.c
index bff7c5a..6b131bc 100644
--- a/common/rb_tree.c
+++ b/common/rb_tree.c
@@ -447,7 +447,7 @@ 

[SCM] CTDB repository - branch 1.13 updated - ctdb-1.43-12-gba94ccc

2012-05-29 Thread Ronnie Sahlberg
The branch, 1.13 has been updated
   via  ba94a0a9ccad7c1de0939e74f0163ae41102 (commit)
  from  33a1661443d90ff4e72c219560a8c6a69c679bb5 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=1.13


- Log -
commit ba94a0a9ccad7c1de0939e74f0163ae41102
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Wed May 30 11:50:13 2012 +1000

Run the shutdown eventscript before we tear down the transport

This allows eventscripts to still be able to call and use ctdb during the 
shutdown phase.

---

Summary of changes:
 server/ctdb_control.c |2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)


Changeset truncated at 500 lines:

diff --git a/server/ctdb_control.c b/server/ctdb_control.c
index 2a16a23..3268877 100644
--- a/server/ctdb_control.c
+++ b/server/ctdb_control.c
@@ -323,10 +323,10 @@ static int32_t ctdb_control_dispatch(struct ctdb_context 
*ctdb,
ctdb_stop_keepalive(ctdb);
ctdb_stop_monitoring(ctdb);
ctdb_release_all_ips(ctdb);
+   ctdb_event_script(ctdb, CTDB_EVENT_SHUTDOWN);
if (ctdb-methods != NULL) {
ctdb-methods-shutdown(ctdb);
}
-   ctdb_event_script(ctdb, CTDB_EVENT_SHUTDOWN);
exit(0);
 
case CTDB_CONTROL_TAKEOVER_IPv4:


-- 
CTDB repository


[SCM] CTDB repository - branch master updated - ctdb-1.13-194-g1a6a011

2012-05-29 Thread Ronnie Sahlberg
The branch, master has been updated
   via  1a6a011c772f7d302d114d7c8a151fa7820ec85f (commit)
  from  ac89da4eea98fa686408c5671a6c44c0fd1d7a58 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=master


- Log -
commit 1a6a011c772f7d302d114d7c8a151fa7820ec85f
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Wed May 30 11:50:13 2012 +1000

Run the shutdown eventscript before we tear down the transport

This allows eventscripts to still be able to call and use ctdb during the 
shutdown phase.

---

Summary of changes:
 server/ctdb_control.c |2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)


Changeset truncated at 500 lines:

diff --git a/server/ctdb_control.c b/server/ctdb_control.c
index 5c1eedd..9708acd 100644
--- a/server/ctdb_control.c
+++ b/server/ctdb_control.c
@@ -323,10 +323,10 @@ static int32_t ctdb_control_dispatch(struct ctdb_context 
*ctdb,
ctdb_stop_keepalive(ctdb);
ctdb_stop_monitoring(ctdb);
ctdb_release_all_ips(ctdb);
+   ctdb_event_script(ctdb, CTDB_EVENT_SHUTDOWN);
if (ctdb-methods != NULL) {
ctdb-methods-shutdown(ctdb);
}
-   ctdb_event_script(ctdb, CTDB_EVENT_SHUTDOWN);
exit(0);
 
case CTDB_CONTROL_TAKEOVER_IPv4:


-- 
CTDB repository


[SCM] CTDB repository - branch 1.13 updated - ctdb-1.43-11-g33a1661

2012-05-27 Thread Ronnie Sahlberg
The branch, 1.13 has been updated
   via  33a1661443d90ff4e72c219560a8c6a69c679bb5 (commit)
   via  978d3f01aaacba723f8cda4eef175d59ae7e561c (commit)
   via  c19533ba5235c3a6ab7ae9f415f1980b6d4dff0e (commit)
   via  21edc6564ea1502b1b542a013bade3eb6c326610 (commit)
   via  7684c48622856435f57c19bc1ba864a23cb7fe5f (commit)
   via  5a04cbb5bca9f739b11867f5bac150544b28b39e (commit)
  from  c8886ad41c80c45619d5eb6e2f95d652b171ca1e (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=1.13


- Log -
commit 33a1661443d90ff4e72c219560a8c6a69c679bb5
Author: Martin Schwenke mar...@meltin.net
Date:   Fri May 25 11:44:56 2012 +1000

tests/eventscripts: Tweak expected output for lockd:b restart

Commit 13acd58c41fba1a33894fbd654fed69ea0eac322 mades this test fail,
since lockd:b and lockd:bs were incorrectly producing the same output.

commit 978d3f01aaacba723f8cda4eef175d59ae7e561c
Author: Martin Schwenke mar...@meltin.net
Date:   Wed May 23 15:36:01 2012 +1000

tests: Complex tests must not be run from a cluster node

Tickle tests fail if run from a node involved in the test.

The condition is actually weaker than this: the test can't be run from
a CTDB node that is hosting public addresses that may be used by the
test.

Rework ctdb_test_check_real_cluster() to support checking this.

Signed-off-by: Martin Schwenke mar...@meltin.net

commit c19533ba5235c3a6ab7ae9f415f1980b6d4dff0e
Author: Martin Schwenke mar...@meltin.net
Date:   Wed May 23 14:24:40 2012 +1000

Eventscripts: Fix deprecated iptables ! usage

This currently causes warning in the logs.

This change is not SLES10-compatible but we already have some other
non-SLES10-compatible changes.

Signed-off-by: Martin Schwenke mar...@meltin.net

commit 21edc6564ea1502b1b542a013bade3eb6c326610
Author: Martin Schwenke mar...@meltin.net
Date:   Tue May 22 11:24:05 2012 +1000

tests: test_wrap needs to set TEST_BIN_DIR when installed

Signed-off-by: Martin Schwenke mar...@meltin.net

commit 7684c48622856435f57c19bc1ba864a23cb7fe5f
Author: Amitay Isaacs ami...@gmail.com
Date:   Fri May 18 12:59:41 2012 +1000

packaging: make ctdb-tests package depend on nc

Signed-off-by: Amitay Isaacs ami...@gmail.com

commit 5a04cbb5bca9f739b11867f5bac150544b28b39e
Author: Amitay Isaacs ami...@gmail.com
Date:   Thu May 10 16:59:39 2012 +1000

tests: Use per node log files when running tests with local daemons

Signed-off-by: Amitay Isaacs ami...@gmail.com

---

Summary of changes:
 config/events.d/11.natgw|4 ++--
 packaging/RPM/ctdb.spec.in  |1 +
 tests/INSTALL   |2 ++
 tests/complex/scripts/local.bash|   11 ++-
 tests/eventscripts/scripts/local.sh |2 +-
 tests/scripts/integration.bash  |2 +-
 6 files changed, 13 insertions(+), 9 deletions(-)


Changeset truncated at 500 lines:

diff --git a/config/events.d/11.natgw b/config/events.d/11.natgw
index ba6d7a5..5093752 100755
--- a/config/events.d/11.natgw
+++ b/config/events.d/11.natgw
@@ -28,7 +28,7 @@ delete_all() {
 
# Delete the masquerading setup from a previous iteration where we
# were the NAT-GW
-   iptables -D POSTROUTING -t nat -s $CTDB_NATGW_PRIVATE_NETWORK -d ! 
$CTDB_NATGW_PRIVATE_NETWORK -j MASQUERADE /dev/null 2/dev/null
+   iptables -D POSTROUTING -t nat -s $CTDB_NATGW_PRIVATE_NETWORK ! -d 
$CTDB_NATGW_PRIVATE_NETWORK -j MASQUERADE /dev/null 2/dev/null
 
# remove any iptables rule we may have on this address
iptables -D INPUT -p tcp --syn -d $_ip/32 -j REJECT 2/dev/null
@@ -71,7 +71,7 @@ case $1 in
if [ $MYPNN = $NATGWMASTER ]; then
# This is the first node, set it up as the NAT GW
echo 1 /proc/sys/net/ipv4/ip_forward
-   iptables -A POSTROUTING -t nat -s $CTDB_NATGW_PRIVATE_NETWORK 
-d ! $CTDB_NATGW_PRIVATE_NETWORK -j MASQUERADE
+   iptables -A POSTROUTING -t nat -s $CTDB_NATGW_PRIVATE_NETWORK ! 
-d $CTDB_NATGW_PRIVATE_NETWORK -j MASQUERADE
 
# block all incoming connections to the natgw address
CTDB_NATGW_PUBLIC_IP_HOST=`echo $CTDB_NATGW_PUBLIC_IP | sed -e 
s/\/.*/\/32/`
diff --git a/packaging/RPM/ctdb.spec.in b/packaging/RPM/ctdb.spec.in
index 6d5b4a4..33da7e5 100644
--- a/packaging/RPM/ctdb.spec.in
+++ b/packaging/RPM/ctdb.spec.in
@@ -182,6 +182,7 @@ development libraries for ctdb
 Summary: CTDB test suite
 Group: Development/Tools
 Requires: ctdb = %{version}
+Requires: nc
 
 %description tests
 test suite for ctdb
diff --git a/tests/INSTALL b/tests/INSTALL
index c67d76d..8c3f777 100755
--- a/tests/INSTALL
+++ b/tests/INSTALL
@@ -74,6 +74,8 @@ done
 for i in events.d functions ; do
 ln -sf ${etcdir}/ctdb/${i} 

[SCM] CTDB repository - branch master updated - ctdb-1.13-189-gfd3b73d

2012-05-27 Thread Ronnie Sahlberg
The branch, master has been updated
   via  fd3b73d7e634f16cbb99d7d5a548e12f00d1aadb (commit)
   via  14012781c3751a514055df29ea70adfb12ecb2d9 (commit)
   via  7640352c6697f9d4e0d13afbc8523afc64e7d462 (commit)
   via  c5e3e4bccbde349739b90d8761e1aa19637887a8 (commit)
   via  d0b539c4d2d4dc8c9eb95801bff09c3bcbeebca5 (commit)
   via  61df417821762d87ed01a7b5e64c35079940344d (commit)
  from  03fa2a517247eb2adfba67248e2466f17ea14418 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=master


- Log -
commit fd3b73d7e634f16cbb99d7d5a548e12f00d1aadb
Author: Martin Schwenke mar...@meltin.net
Date:   Fri May 25 11:44:56 2012 +1000

tests/eventscripts: Tweak expected output for lockd:b restart

Commit 13acd58c41fba1a33894fbd654fed69ea0eac322 mades this test fail,
since lockd:b and lockd:bs were incorrectly producing the same output.

commit 14012781c3751a514055df29ea70adfb12ecb2d9
Author: Martin Schwenke mar...@meltin.net
Date:   Wed May 23 15:36:01 2012 +1000

tests: Complex tests must not be run from a cluster node

Tickle tests fail if run from a node involved in the test.

The condition is actually weaker than this: the test can't be run from
a CTDB node that is hosting public addresses that may be used by the
test.

Rework ctdb_test_check_real_cluster() to support checking this.

Signed-off-by: Martin Schwenke mar...@meltin.net

commit 7640352c6697f9d4e0d13afbc8523afc64e7d462
Author: Martin Schwenke mar...@meltin.net
Date:   Wed May 23 14:24:40 2012 +1000

Eventscripts: Fix deprecated iptables ! usage

This currently causes warning in the logs.

This change is not SLES10-compatible but we already have some other
non-SLES10-compatible changes.

Signed-off-by: Martin Schwenke mar...@meltin.net

commit c5e3e4bccbde349739b90d8761e1aa19637887a8
Author: Martin Schwenke mar...@meltin.net
Date:   Tue May 22 11:24:05 2012 +1000

tests: test_wrap needs to set TEST_BIN_DIR when installed

Signed-off-by: Martin Schwenke mar...@meltin.net

commit d0b539c4d2d4dc8c9eb95801bff09c3bcbeebca5
Author: Amitay Isaacs ami...@gmail.com
Date:   Fri May 18 12:59:41 2012 +1000

packaging: make ctdb-tests package depend on nc

Signed-off-by: Amitay Isaacs ami...@gmail.com

commit 61df417821762d87ed01a7b5e64c35079940344d
Author: Amitay Isaacs ami...@gmail.com
Date:   Thu May 10 16:59:39 2012 +1000

tests: Use per node log files when running tests with local daemons

Signed-off-by: Amitay Isaacs ami...@gmail.com

---

Summary of changes:
 config/events.d/11.natgw|4 ++--
 packaging/RPM/ctdb.spec.in  |1 +
 tests/INSTALL   |2 ++
 tests/complex/scripts/local.bash|   11 ++-
 tests/eventscripts/scripts/local.sh |2 +-
 tests/scripts/integration.bash  |2 +-
 6 files changed, 13 insertions(+), 9 deletions(-)


Changeset truncated at 500 lines:

diff --git a/config/events.d/11.natgw b/config/events.d/11.natgw
index ba6d7a5..5093752 100755
--- a/config/events.d/11.natgw
+++ b/config/events.d/11.natgw
@@ -28,7 +28,7 @@ delete_all() {
 
# Delete the masquerading setup from a previous iteration where we
# were the NAT-GW
-   iptables -D POSTROUTING -t nat -s $CTDB_NATGW_PRIVATE_NETWORK -d ! 
$CTDB_NATGW_PRIVATE_NETWORK -j MASQUERADE /dev/null 2/dev/null
+   iptables -D POSTROUTING -t nat -s $CTDB_NATGW_PRIVATE_NETWORK ! -d 
$CTDB_NATGW_PRIVATE_NETWORK -j MASQUERADE /dev/null 2/dev/null
 
# remove any iptables rule we may have on this address
iptables -D INPUT -p tcp --syn -d $_ip/32 -j REJECT 2/dev/null
@@ -71,7 +71,7 @@ case $1 in
if [ $MYPNN = $NATGWMASTER ]; then
# This is the first node, set it up as the NAT GW
echo 1 /proc/sys/net/ipv4/ip_forward
-   iptables -A POSTROUTING -t nat -s $CTDB_NATGW_PRIVATE_NETWORK 
-d ! $CTDB_NATGW_PRIVATE_NETWORK -j MASQUERADE
+   iptables -A POSTROUTING -t nat -s $CTDB_NATGW_PRIVATE_NETWORK ! 
-d $CTDB_NATGW_PRIVATE_NETWORK -j MASQUERADE
 
# block all incoming connections to the natgw address
CTDB_NATGW_PUBLIC_IP_HOST=`echo $CTDB_NATGW_PUBLIC_IP | sed -e 
s/\/.*/\/32/`
diff --git a/packaging/RPM/ctdb.spec.in b/packaging/RPM/ctdb.spec.in
index 09a30f4..381739c 100644
--- a/packaging/RPM/ctdb.spec.in
+++ b/packaging/RPM/ctdb.spec.in
@@ -182,6 +182,7 @@ development libraries for ctdb
 Summary: CTDB test suite
 Group: Development/Tools
 Requires: ctdb = %{version}
+Requires: nc
 
 %description tests
 test suite for ctdb
diff --git a/tests/INSTALL b/tests/INSTALL
index c67d76d..8c3f777 100755
--- a/tests/INSTALL
+++ b/tests/INSTALL
@@ -74,6 +74,8 @@ done
 for i in events.d functions ; do
 ln -sf ${etcdir}/ctdb/${i} 

[SCM] CTDB repository - branch 1.13 updated - ctdb-1.43-5-gc8886ad

2012-05-24 Thread Ronnie Sahlberg
The branch, 1.13 has been updated
   via  c8886ad41c80c45619d5eb6e2f95d652b171ca1e (commit)
   via  ea073ef2f73343247e653c7c39f2e0f5e34a0c39 (commit)
  from  b8260448c192c3866bffb99b475a4b3de57f38b3 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=1.13


- Log -
commit c8886ad41c80c45619d5eb6e2f95d652b171ca1e
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Fri May 25 12:31:11 2012 +1000

RECOVERY: Increase the time we allow before timing out recovery related 
tasks.

If the system is temporarily taking unusually long to perform these tasks 
it is better to wait a lot longer and allow the tasks to complete than timing 
out repeatedly and then becomming banned.

commit ea073ef2f73343247e653c7c39f2e0f5e34a0c39
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Fri May 25 12:27:59 2012 +1000

RECOVER: When we pull databases during recovery, we used to reallocate the 
databuffer for each entry added. This would normally not be an issue, but for 
cases where memory is fragmented, this could start to cost significant cpu if 
we need to reallocate and move to a different region.

Change this to instead preallocate , by default, 10MByte chunks to the data 
buffer.
This significantly reduces the number of potential reallocate and move  
operations that may be required.

Create a tunable to override/change how much preallocation should be used.

---

Summary of changes:
 include/ctdb_private.h |1 +
 server/ctdb_recover.c  |7 ++-
 server/ctdb_recoverd.c |7 ++-
 server/ctdb_tunables.c |5 +++--
 4 files changed, 16 insertions(+), 4 deletions(-)


Changeset truncated at 500 lines:

diff --git a/include/ctdb_private.h b/include/ctdb_private.h
index 0f494b4..7c3fdf0 100644
--- a/include/ctdb_private.h
+++ b/include/ctdb_private.h
@@ -134,6 +134,7 @@ struct ctdb_tunable {
uint32_t db_record_count_warn;
uint32_t db_record_size_warn;
uint32_t db_size_warn;
+   uint32_t pulldb_preallocation_size;
 };
 
 /*
diff --git a/server/ctdb_recover.c b/server/ctdb_recover.c
index 05f72f9..e54312f 100644
--- a/server/ctdb_recover.c
+++ b/server/ctdb_recover.c
@@ -348,6 +348,7 @@ struct pulldb_data {
struct ctdb_db_context *ctdb_db;
struct ctdb_marshall_buffer *pulldata;
uint32_t len;
+   uint32_t allocated_len;
bool failed;
 };
 
@@ -364,7 +365,10 @@ static int traverse_pulldb(struct tdb_context *tdb, 
TDB_DATA key, TDB_DATA data,
params-failed = true;
return -1;
}
-   params-pulldata = talloc_realloc_size(NULL, params-pulldata, 
rec-length + params-len);
+   if (params-len + rec-length = params-allocated_len) {
+   params-allocated_len = rec-length + params-len + 
ctdb-tunable.pulldb_preallocation_size;
+   params-pulldata = talloc_realloc_size(NULL, params-pulldata, 
params-allocated_len);
+   }
if (params-pulldata == NULL) {
DEBUG(DEBUG_CRIT,(__location__  Failed to expand pulldb_data 
to %u\n, rec-length + params-len));
ctdb_fatal(params-ctdb, failed to allocate memory for 
recovery. shutting down\n);
@@ -414,6 +418,7 @@ int32_t ctdb_control_pull_db(struct ctdb_context *ctdb, 
TDB_DATA indata, TDB_DAT
params.ctdb_db = ctdb_db;
params.pulldata = reply;
params.len = offsetof(struct ctdb_marshall_buffer, data);
+   params.allocated_len = params.len;
params.failed = false;
 
if (ctdb_db-unhealthy_reason) {
diff --git a/server/ctdb_recoverd.c b/server/ctdb_recoverd.c
index f739900..b380746 100644
--- a/server/ctdb_recoverd.c
+++ b/server/ctdb_recoverd.c
@@ -1178,6 +1178,7 @@ struct recdb_data {
struct ctdb_context *ctdb;
struct ctdb_marshall_buffer *recdata;
uint32_t len;
+   uint32_t allocated_len;
bool failed;
bool persistent;
 };
@@ -1206,7 +1207,10 @@ static int traverse_recdb(struct tdb_context *tdb, 
TDB_DATA key, TDB_DATA data,
params-failed = true;
return -1;
}
-   params-recdata = talloc_realloc_size(NULL, params-recdata, 
rec-length + params-len);
+   if (params-len + rec-length = params-allocated_len) {
+   params-allocated_len = rec-length + params-len + 
params-ctdb-tunable.pulldb_preallocation_size;
+   params-recdata = talloc_realloc_size(NULL, params-recdata, 
params-allocated_len);
+   }
if (params-recdata == NULL) {
DEBUG(DEBUG_CRIT,(__location__  Failed to expand recdata to %u 
(%u records)\n, 
 rec-length + params-len, params-recdata-count));
@@ -1245,6 +1249,7 @@ static int push_recdb_database(struct ctdb_context *ctdb, 
uint32_t dbid,
params.ctdb = ctdb;
params.recdata = recdata

[SCM] CTDB repository - branch master updated - ctdb-1.13-183-g03fa2a5

2012-05-24 Thread Ronnie Sahlberg
The branch, master has been updated
   via  03fa2a517247eb2adfba67248e2466f17ea14418 (commit)
   via  1f262deaad0818f159f9c68330f7fec121679023 (commit)
  from  6cf6a9b071bd8dd730717ca07ff73bf247bb (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=master


- Log -
commit 03fa2a517247eb2adfba67248e2466f17ea14418
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Fri May 25 12:31:11 2012 +1000

RECOVERY: Increase the time we allow before timing out recovery related 
tasks.

If the system is temporarily taking unusually long to perform these tasks 
it is better to wait a lot longer and allow the tasks to complete than timing 
out repeatedly and then becomming banned.

commit 1f262deaad0818f159f9c68330f7fec121679023
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Fri May 25 12:27:59 2012 +1000

RECOVER: When we pull databases during recovery, we used to reallocate the 
databuffer for each entry added. This would normally not be an issue, but for 
cases where memory is fragmented, this could start to cost significant cpu if 
we need to reallocate and move to a different region.

Change this to instead preallocate , by default, 10MByte chunks to the data 
buffer.
This significantly reduces the number of potential reallocate and move  
operations that may be required.

Create a tunable to override/change how much preallocation should be used.

---

Summary of changes:
 include/ctdb_private.h |1 +
 server/ctdb_recover.c  |7 ++-
 server/ctdb_recoverd.c |7 ++-
 server/ctdb_tunables.c |5 +++--
 4 files changed, 16 insertions(+), 4 deletions(-)


Changeset truncated at 500 lines:

diff --git a/include/ctdb_private.h b/include/ctdb_private.h
index 0f494b4..7c3fdf0 100644
--- a/include/ctdb_private.h
+++ b/include/ctdb_private.h
@@ -134,6 +134,7 @@ struct ctdb_tunable {
uint32_t db_record_count_warn;
uint32_t db_record_size_warn;
uint32_t db_size_warn;
+   uint32_t pulldb_preallocation_size;
 };
 
 /*
diff --git a/server/ctdb_recover.c b/server/ctdb_recover.c
index 05f72f9..e54312f 100644
--- a/server/ctdb_recover.c
+++ b/server/ctdb_recover.c
@@ -348,6 +348,7 @@ struct pulldb_data {
struct ctdb_db_context *ctdb_db;
struct ctdb_marshall_buffer *pulldata;
uint32_t len;
+   uint32_t allocated_len;
bool failed;
 };
 
@@ -364,7 +365,10 @@ static int traverse_pulldb(struct tdb_context *tdb, 
TDB_DATA key, TDB_DATA data,
params-failed = true;
return -1;
}
-   params-pulldata = talloc_realloc_size(NULL, params-pulldata, 
rec-length + params-len);
+   if (params-len + rec-length = params-allocated_len) {
+   params-allocated_len = rec-length + params-len + 
ctdb-tunable.pulldb_preallocation_size;
+   params-pulldata = talloc_realloc_size(NULL, params-pulldata, 
params-allocated_len);
+   }
if (params-pulldata == NULL) {
DEBUG(DEBUG_CRIT,(__location__  Failed to expand pulldb_data 
to %u\n, rec-length + params-len));
ctdb_fatal(params-ctdb, failed to allocate memory for 
recovery. shutting down\n);
@@ -414,6 +418,7 @@ int32_t ctdb_control_pull_db(struct ctdb_context *ctdb, 
TDB_DATA indata, TDB_DAT
params.ctdb_db = ctdb_db;
params.pulldata = reply;
params.len = offsetof(struct ctdb_marshall_buffer, data);
+   params.allocated_len = params.len;
params.failed = false;
 
if (ctdb_db-unhealthy_reason) {
diff --git a/server/ctdb_recoverd.c b/server/ctdb_recoverd.c
index f739900..b380746 100644
--- a/server/ctdb_recoverd.c
+++ b/server/ctdb_recoverd.c
@@ -1178,6 +1178,7 @@ struct recdb_data {
struct ctdb_context *ctdb;
struct ctdb_marshall_buffer *recdata;
uint32_t len;
+   uint32_t allocated_len;
bool failed;
bool persistent;
 };
@@ -1206,7 +1207,10 @@ static int traverse_recdb(struct tdb_context *tdb, 
TDB_DATA key, TDB_DATA data,
params-failed = true;
return -1;
}
-   params-recdata = talloc_realloc_size(NULL, params-recdata, 
rec-length + params-len);
+   if (params-len + rec-length = params-allocated_len) {
+   params-allocated_len = rec-length + params-len + 
params-ctdb-tunable.pulldb_preallocation_size;
+   params-recdata = talloc_realloc_size(NULL, params-recdata, 
params-allocated_len);
+   }
if (params-recdata == NULL) {
DEBUG(DEBUG_CRIT,(__location__  Failed to expand recdata to %u 
(%u records)\n, 
 rec-length + params-len, params-recdata-count));
@@ -1245,6 +1249,7 @@ static int push_recdb_database(struct ctdb_context *ctdb, 
uint32_t dbid,
params.ctdb = ctdb;
params.recdata = recdata

[SCM] CTDB repository - branch 1.13 updated - ctdb-1.43-1-gb5f260d

2012-05-20 Thread Ronnie Sahlberg
The branch, 1.13 has been updated
   via  b5f260d84b3989c5da5afb3010861f838dc59f1a (commit)
  from  f66d2b30eedc755cb8bc8b1c1e18141c6ee0c946 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=1.13


- Log -
commit b5f260d84b3989c5da5afb3010861f838dc59f1a
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Mon May 21 09:17:05 2012 +1000

TEVENT: Add back tracking of long runnig  events to the local copy of 
tevent library

---

Summary of changes:
 lib/tevent/tevent_epoll.c|4 +++
 lib/tevent/tevent_internal.h |2 +
 lib/tevent/tevent_select.c   |4 +++
 lib/tevent/tevent_standard.c |4 +++
 lib/tevent/tevent_util.c |   52 ++
 5 files changed, 66 insertions(+), 0 deletions(-)


Changeset truncated at 500 lines:

diff --git a/lib/tevent/tevent_epoll.c b/lib/tevent/tevent_epoll.c
index f5a69dd..7c09dd7 100644
--- a/lib/tevent/tevent_epoll.c
+++ b/lib/tevent/tevent_epoll.c
@@ -242,6 +242,8 @@ static void epoll_change_event(struct epoll_event_context 
*epoll_ev, struct teve
}
 }
 
+extern pid_t ctdbd_pid;
+
 /*
   event loop handling using epoll
 */
@@ -264,7 +266,9 @@ static int epoll_event_loop(struct epoll_event_context 
*epoll_ev, struct timeval
return 0;
}
 
+   if (getpid() == ctdbd_pid) tevent_before_wait(epoll_ev-ev);
ret = epoll_wait(epoll_ev-epoll_fd, events, MAXEVENTS, timeout);
+   if (getpid() == ctdbd_pid) tevent_after_wait(epoll_ev-ev);
 
if (ret == -1  errno == EINTR  epoll_ev-ev-signal_events) {
if (tevent_common_check_signal(epoll_ev-ev)) {
diff --git a/lib/tevent/tevent_internal.h b/lib/tevent/tevent_internal.h
index 7f1d876..707bfab 100644
--- a/lib/tevent/tevent_internal.h
+++ b/lib/tevent/tevent_internal.h
@@ -313,3 +313,5 @@ bool tevent_poll_init(void);
 #ifdef HAVE_EPOLL
 bool tevent_epoll_init(void);
 #endif
+void tevent_before_wait(struct tevent_context *ev);
+void tevent_after_wait(struct tevent_context *ev);
diff --git a/lib/tevent/tevent_select.c b/lib/tevent/tevent_select.c
index 51c1dec..6fa38ce 100644
--- a/lib/tevent/tevent_select.c
+++ b/lib/tevent/tevent_select.c
@@ -130,6 +130,8 @@ static struct tevent_fd *select_event_add_fd(struct 
tevent_context *ev, TALLOC_C
return fde;
 }
 
+extern pid_t ctdbd_pid;
+
 /*
   event loop handling using select()
 */
@@ -167,7 +169,9 @@ static int select_event_loop_select(struct 
select_event_context *select_ev, stru
return 0;
}
 
+   if (getpid() == ctdbd_pid) tevent_before_wait(select_ev-ev);
selrtn = select(select_ev-maxfd+1, r_fds, w_fds, NULL, tvalp);
+   if (getpid() == ctdbd_pid) tevent_after_wait(select_ev-ev);
 
if (selrtn == -1  errno == EINTR 
select_ev-ev-signal_events) {
diff --git a/lib/tevent/tevent_standard.c b/lib/tevent/tevent_standard.c
index 534576c..4041b6a 100644
--- a/lib/tevent/tevent_standard.c
+++ b/lib/tevent/tevent_standard.c
@@ -256,6 +256,8 @@ static void epoll_change_event(struct std_event_context 
*std_ev, struct tevent_f
}
 }
 
+extern pid_t ctdbd_pid;
+
 /*
   event loop handling using epoll
 */
@@ -278,7 +280,9 @@ static int epoll_event_loop(struct std_event_context 
*std_ev, struct timeval *tv
return 0;
}
 
+   if (getpid() == ctdbd_pid) tevent_before_wait(std_ev-ev);
ret = epoll_wait(std_ev-epoll_fd, events, MAXEVENTS, timeout);
+   if (getpid() == ctdbd_pid) tevent_after_wait(std_ev-ev);
 
if (ret == -1  errno == EINTR  std_ev-ev-signal_events) {
if (tevent_common_check_signal(std_ev-ev)) {
diff --git a/lib/tevent/tevent_util.c b/lib/tevent/tevent_util.c
index 16af8f3..e3a1605 100644
--- a/lib/tevent/tevent_util.c
+++ b/lib/tevent/tevent_util.c
@@ -105,3 +105,55 @@ bool ev_set_close_on_exec(int fd)
 #endif
return false;
 }
+
+
+static struct timeval tevent_before_wait_ts;
+static struct timeval tevent_after_wait_ts;
+
+/*
+ * measure the time difference between multiple arrivals
+ * to the point where we wait for new events to come in
+ *
+ * allows to measure how long it takes to work on a 
+ * event
+ */
+void tevent_before_wait(struct tevent_context *ev) {
+
+   struct timeval diff;
+   struct timeval now = tevent_timeval_current();
+
+   if (!tevent_timeval_is_zero(tevent_after_wait_ts)) {
+   diff = tevent_timeval_until(tevent_after_wait_ts, now);
+   if (diff.tv_sec  3) {
+   tevent_debug(ev, TEVENT_DEBUG_ERROR,  __location__ 
+ Handling event took %d seconds!,
+(int) diff.tv_sec);
+   }
+   }
+
+   tevent_before_wait_ts = tevent_timeval_current();
+
+}
+
+/*
+ * measure how long the select()/epoll() call took
+ *
+ * allows

[SCM] CTDB repository - branch master updated - ctdb-1.13-179-g5aba53e

2012-05-20 Thread Ronnie Sahlberg
The branch, master has been updated
   via  5aba53e6adcfcd7edbdac9e30aa5fcba176aca00 (commit)
  from  f59b40b3f8ea3da8ffb8601bc025e83c237072d5 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=master


- Log -
commit 5aba53e6adcfcd7edbdac9e30aa5fcba176aca00
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Mon May 21 09:17:05 2012 +1000

TEVENT: Add back tracking of long runnig  events to the local copy of 
tevent library

---

Summary of changes:
 lib/tevent/tevent_epoll.c|4 +++
 lib/tevent/tevent_internal.h |2 +
 lib/tevent/tevent_select.c   |4 +++
 lib/tevent/tevent_standard.c |4 +++
 lib/tevent/tevent_util.c |   52 ++
 5 files changed, 66 insertions(+), 0 deletions(-)


Changeset truncated at 500 lines:

diff --git a/lib/tevent/tevent_epoll.c b/lib/tevent/tevent_epoll.c
index f5a69dd..7c09dd7 100644
--- a/lib/tevent/tevent_epoll.c
+++ b/lib/tevent/tevent_epoll.c
@@ -242,6 +242,8 @@ static void epoll_change_event(struct epoll_event_context 
*epoll_ev, struct teve
}
 }
 
+extern pid_t ctdbd_pid;
+
 /*
   event loop handling using epoll
 */
@@ -264,7 +266,9 @@ static int epoll_event_loop(struct epoll_event_context 
*epoll_ev, struct timeval
return 0;
}
 
+   if (getpid() == ctdbd_pid) tevent_before_wait(epoll_ev-ev);
ret = epoll_wait(epoll_ev-epoll_fd, events, MAXEVENTS, timeout);
+   if (getpid() == ctdbd_pid) tevent_after_wait(epoll_ev-ev);
 
if (ret == -1  errno == EINTR  epoll_ev-ev-signal_events) {
if (tevent_common_check_signal(epoll_ev-ev)) {
diff --git a/lib/tevent/tevent_internal.h b/lib/tevent/tevent_internal.h
index 7f1d876..707bfab 100644
--- a/lib/tevent/tevent_internal.h
+++ b/lib/tevent/tevent_internal.h
@@ -313,3 +313,5 @@ bool tevent_poll_init(void);
 #ifdef HAVE_EPOLL
 bool tevent_epoll_init(void);
 #endif
+void tevent_before_wait(struct tevent_context *ev);
+void tevent_after_wait(struct tevent_context *ev);
diff --git a/lib/tevent/tevent_select.c b/lib/tevent/tevent_select.c
index 51c1dec..6fa38ce 100644
--- a/lib/tevent/tevent_select.c
+++ b/lib/tevent/tevent_select.c
@@ -130,6 +130,8 @@ static struct tevent_fd *select_event_add_fd(struct 
tevent_context *ev, TALLOC_C
return fde;
 }
 
+extern pid_t ctdbd_pid;
+
 /*
   event loop handling using select()
 */
@@ -167,7 +169,9 @@ static int select_event_loop_select(struct 
select_event_context *select_ev, stru
return 0;
}
 
+   if (getpid() == ctdbd_pid) tevent_before_wait(select_ev-ev);
selrtn = select(select_ev-maxfd+1, r_fds, w_fds, NULL, tvalp);
+   if (getpid() == ctdbd_pid) tevent_after_wait(select_ev-ev);
 
if (selrtn == -1  errno == EINTR 
select_ev-ev-signal_events) {
diff --git a/lib/tevent/tevent_standard.c b/lib/tevent/tevent_standard.c
index 534576c..4041b6a 100644
--- a/lib/tevent/tevent_standard.c
+++ b/lib/tevent/tevent_standard.c
@@ -256,6 +256,8 @@ static void epoll_change_event(struct std_event_context 
*std_ev, struct tevent_f
}
 }
 
+extern pid_t ctdbd_pid;
+
 /*
   event loop handling using epoll
 */
@@ -278,7 +280,9 @@ static int epoll_event_loop(struct std_event_context 
*std_ev, struct timeval *tv
return 0;
}
 
+   if (getpid() == ctdbd_pid) tevent_before_wait(std_ev-ev);
ret = epoll_wait(std_ev-epoll_fd, events, MAXEVENTS, timeout);
+   if (getpid() == ctdbd_pid) tevent_after_wait(std_ev-ev);
 
if (ret == -1  errno == EINTR  std_ev-ev-signal_events) {
if (tevent_common_check_signal(std_ev-ev)) {
diff --git a/lib/tevent/tevent_util.c b/lib/tevent/tevent_util.c
index 16af8f3..e3a1605 100644
--- a/lib/tevent/tevent_util.c
+++ b/lib/tevent/tevent_util.c
@@ -105,3 +105,55 @@ bool ev_set_close_on_exec(int fd)
 #endif
return false;
 }
+
+
+static struct timeval tevent_before_wait_ts;
+static struct timeval tevent_after_wait_ts;
+
+/*
+ * measure the time difference between multiple arrivals
+ * to the point where we wait for new events to come in
+ *
+ * allows to measure how long it takes to work on a 
+ * event
+ */
+void tevent_before_wait(struct tevent_context *ev) {
+
+   struct timeval diff;
+   struct timeval now = tevent_timeval_current();
+
+   if (!tevent_timeval_is_zero(tevent_after_wait_ts)) {
+   diff = tevent_timeval_until(tevent_after_wait_ts, now);
+   if (diff.tv_sec  3) {
+   tevent_debug(ev, TEVENT_DEBUG_ERROR,  __location__ 
+ Handling event took %d seconds!,
+(int) diff.tv_sec);
+   }
+   }
+
+   tevent_before_wait_ts = tevent_timeval_current();
+
+}
+
+/*
+ * measure how long the select()/epoll() call took

[SCM] CTDB repository - branch 1.13 updated - ctdb-1.43-3-gb826044

2012-05-20 Thread Ronnie Sahlberg
The branch, 1.13 has been updated
   via  b8260448c192c3866bffb99b475a4b3de57f38b3 (commit)
   via  4cd124c5759771c18c3277bf83aa277373b43942 (commit)
  from  b5f260d84b3989c5da5afb3010861f838dc59f1a (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=1.13


- Log -
commit b8260448c192c3866bffb99b475a4b3de57f38b3
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Mon May 21 14:01:04 2012 +1000

DOCS: Document the new tunables to produce warnings if databases grow 
unexpectedly big.

commit 4cd124c5759771c18c3277bf83aa277373b43942
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Mon May 21 13:11:38 2012 +1000

DEBUG: Add checks for and print debug messages when 1) a database contains 
very many records, 2) when a database is very big, 3) when a single record is 
very big.

Add tunables to control when to log these instances and allow it to be 
completely turned off by setting the threshold to 0

---

Summary of changes:
 doc/ctdbd.1|   21 +-
 doc/ctdbd.1.html   |  163 ++--
 doc/ctdbd.1.xml|   28 
 include/ctdb_private.h |3 +
 server/ctdb_recover.c  |   17 +
 server/ctdb_tunables.c |5 +-
 6 files changed, 158 insertions(+), 79 deletions(-)


Changeset truncated at 500 lines:

diff --git a/doc/ctdbd.1 b/doc/ctdbd.1
index be527aa..60abf03 100644
--- a/doc/ctdbd.1
+++ b/doc/ctdbd.1
@@ -1,13 +1,13 @@
 '\ t
 .\ Title: ctdbd
 .\Author: [FIXME: author] [see http://docbook.sf.net/el/author]
-.\ Generator: DocBook XSL Stylesheets v1.76.1 http://docbook.sf.net/
-.\  Date: 03/23/2012
+.\ Generator: DocBook XSL Stylesheets v1.75.2 http://docbook.sf.net/
+.\  Date: 05/21/2012
 .\Manual: CTDB - clustered TDB database
 .\Source: ctdb
 .\  Language: English
 .\
-.TH CTDBD 1 03/23/2012 ctdb CTDB \- clustered TDB database
+.TH CTDBD 1 05/21/2012 ctdb CTDB \- clustered TDB database
 .\ -
 .\ * Define some portability stuff
 .\ -
@@ -483,6 +483,21 @@ When you enable this tunable, CTDB will no longer attempt 
to recover the cluster
 Default: 0
 .PP
 When set to 1, ctdb will allow ip addresses to be failed over onto this 
node\. Any ip addresses that the node currently hosts will remain on the node 
but no new ip addresses can be failed over onto the node\.
+.SS DBRecordCountWarn
+.PP
+Default: 10
+.PP
+When set to non\-zero, ctdb will log a warning when we try to recover a 
database with more than this many records\. This will produce a warning if a 
database grows uncontrollably with orphaned records\.
+.SS DBRecordSizeWarn
+.PP
+Default: 1000
+.PP
+When set to non\-zero, ctdb will log a warning when we try to recover a 
database where a single record is bigger than this\. This will produce a 
warning if a database record grows uncontrollably with orphaned sub\-records\.
+.SS DBSizeWarn
+.PP
+Default: 10
+.PP
+When set to non\-zero, ctdb will log a warning when we try to recover a 
database bigger than this\. This will produce a warning if a database grows 
uncontrollably\.
 .SS VerboseMemoryNames
 .PP
 Default: 0
diff --git a/doc/ctdbd.1.html b/doc/ctdbd.1.html
index 95f1e75..0530b22 100644
--- a/doc/ctdbd.1.html
+++ b/doc/ctdbd.1.html
@@ -1,4 +1,4 @@
-htmlheadmeta http-equiv=Content-Type content=text/html; 
charset=ISO-8859-1titlectdbd/titlemeta name=generator content=DocBook 
XSL Stylesheets V1.76.1/headbody bgcolor=white text=black 
link=#FF vlink=#840084 alink=#FFdiv class=refentry 
title=ctdbda name=ctdbd.1/adiv class=titlepage/divdiv 
class=refnamedivh2Name/h2pctdbd #8212; The CTDB cluster 
daemon/p/divdiv class=refsynopsisdiv 
title=Synopsish2Synopsis/h2div class=cmdsynopsispcode 
class=commandctdbd/code /p/divdiv class=cmdsynopsispcode 
class=commandctdbd/code  [-? --help] [-d --debug=lt;INTEGERgt;] 
{--dbdir=lt;directorygt;} {--dbdir-persistent=lt;directorygt;} 
[--event-script-dir=lt;directorygt;] [-i --interactive] 
[--listen=lt;addressgt;] [--logfile=lt;filenamegt;] [--lvs] 
{--nlist=lt;filenamegt;} [--no-lmaster] [--no-recmaster] [--nosetsched] 
{--notification-script=lt;filenamegt;} [--public-add
 resses=lt;filenamegt;] [--public-interface=lt;interfacegt;] 
{--reclock=lt;filenamegt;} [--single-public-ip=lt;addressgt;] 
[--socket=lt;filenamegt;] [--start-as-disabled] [--start-as-stopped] 
[--syslog] [--log-ringbuf-size=lt;num-entriesgt;] [--torture] 
[--transport=lt;STRINGgt;] [--usage]/p/div/divdiv class=refsect1 
title=DESCRIPTIONa name=idp187688/ah2DESCRIPTION/h2p
+htmlheadmeta http-equiv=Content-Type content=text/html; 
charset=ISO-8859-1titlectdbd/titlemeta name=generator content=DocBook 
XSL Stylesheets V1.75.2/headbody bgcolor=white text=black 
link=#FF vlink

[SCM] CTDB repository - branch master updated - ctdb-1.13-181-g6cf6a9b

2012-05-20 Thread Ronnie Sahlberg
The branch, master has been updated
   via  6cf6a9b071bd8dd730717ca07ff73bf247bb (commit)
   via  9ed58fef4991725f75509433496f4d5ffae0ae87 (commit)
  from  5aba53e6adcfcd7edbdac9e30aa5fcba176aca00 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=master


- Log -
commit 6cf6a9b071bd8dd730717ca07ff73bf247bb
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Mon May 21 14:01:04 2012 +1000

DOCS: Document the new tunables to produce warnings if databases grow 
unexpectedly big.

commit 9ed58fef4991725f75509433496f4d5ffae0ae87
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Mon May 21 13:11:38 2012 +1000

DEBUG: Add checks for and print debug messages when 1) a database contains 
very many records, 2) when a database is very big, 3) when a single record is 
very big.

Add tunables to control when to log these instances and allow it to be 
completely turned off by setting the threshold to 0

---

Summary of changes:
 doc/ctdbd.1|   21 +-
 doc/ctdbd.1.html   |  163 ++--
 doc/ctdbd.1.xml|   28 
 include/ctdb_private.h |3 +
 server/ctdb_recover.c  |   17 +
 server/ctdb_tunables.c |5 +-
 6 files changed, 158 insertions(+), 79 deletions(-)


Changeset truncated at 500 lines:

diff --git a/doc/ctdbd.1 b/doc/ctdbd.1
index be527aa..60abf03 100644
--- a/doc/ctdbd.1
+++ b/doc/ctdbd.1
@@ -1,13 +1,13 @@
 '\ t
 .\ Title: ctdbd
 .\Author: [FIXME: author] [see http://docbook.sf.net/el/author]
-.\ Generator: DocBook XSL Stylesheets v1.76.1 http://docbook.sf.net/
-.\  Date: 03/23/2012
+.\ Generator: DocBook XSL Stylesheets v1.75.2 http://docbook.sf.net/
+.\  Date: 05/21/2012
 .\Manual: CTDB - clustered TDB database
 .\Source: ctdb
 .\  Language: English
 .\
-.TH CTDBD 1 03/23/2012 ctdb CTDB \- clustered TDB database
+.TH CTDBD 1 05/21/2012 ctdb CTDB \- clustered TDB database
 .\ -
 .\ * Define some portability stuff
 .\ -
@@ -483,6 +483,21 @@ When you enable this tunable, CTDB will no longer attempt 
to recover the cluster
 Default: 0
 .PP
 When set to 1, ctdb will allow ip addresses to be failed over onto this 
node\. Any ip addresses that the node currently hosts will remain on the node 
but no new ip addresses can be failed over onto the node\.
+.SS DBRecordCountWarn
+.PP
+Default: 10
+.PP
+When set to non\-zero, ctdb will log a warning when we try to recover a 
database with more than this many records\. This will produce a warning if a 
database grows uncontrollably with orphaned records\.
+.SS DBRecordSizeWarn
+.PP
+Default: 1000
+.PP
+When set to non\-zero, ctdb will log a warning when we try to recover a 
database where a single record is bigger than this\. This will produce a 
warning if a database record grows uncontrollably with orphaned sub\-records\.
+.SS DBSizeWarn
+.PP
+Default: 10
+.PP
+When set to non\-zero, ctdb will log a warning when we try to recover a 
database bigger than this\. This will produce a warning if a database grows 
uncontrollably\.
 .SS VerboseMemoryNames
 .PP
 Default: 0
diff --git a/doc/ctdbd.1.html b/doc/ctdbd.1.html
index 95f1e75..0530b22 100644
--- a/doc/ctdbd.1.html
+++ b/doc/ctdbd.1.html
@@ -1,4 +1,4 @@
-htmlheadmeta http-equiv=Content-Type content=text/html; 
charset=ISO-8859-1titlectdbd/titlemeta name=generator content=DocBook 
XSL Stylesheets V1.76.1/headbody bgcolor=white text=black 
link=#FF vlink=#840084 alink=#FFdiv class=refentry 
title=ctdbda name=ctdbd.1/adiv class=titlepage/divdiv 
class=refnamedivh2Name/h2pctdbd #8212; The CTDB cluster 
daemon/p/divdiv class=refsynopsisdiv 
title=Synopsish2Synopsis/h2div class=cmdsynopsispcode 
class=commandctdbd/code /p/divdiv class=cmdsynopsispcode 
class=commandctdbd/code  [-? --help] [-d --debug=lt;INTEGERgt;] 
{--dbdir=lt;directorygt;} {--dbdir-persistent=lt;directorygt;} 
[--event-script-dir=lt;directorygt;] [-i --interactive] 
[--listen=lt;addressgt;] [--logfile=lt;filenamegt;] [--lvs] 
{--nlist=lt;filenamegt;} [--no-lmaster] [--no-recmaster] [--nosetsched] 
{--notification-script=lt;filenamegt;} [--public-add
 resses=lt;filenamegt;] [--public-interface=lt;interfacegt;] 
{--reclock=lt;filenamegt;} [--single-public-ip=lt;addressgt;] 
[--socket=lt;filenamegt;] [--start-as-disabled] [--start-as-stopped] 
[--syslog] [--log-ringbuf-size=lt;num-entriesgt;] [--torture] 
[--transport=lt;STRINGgt;] [--usage]/p/div/divdiv class=refsect1 
title=DESCRIPTIONa name=idp187688/ah2DESCRIPTION/h2p
+htmlheadmeta http-equiv=Content-Type content=text/html; 
charset=ISO-8859-1titlectdbd/titlemeta name=generator content=DocBook 
XSL Stylesheets V1.75.2/headbody bgcolor=white text=black 
link=#FF vlink

[SCM] CTDB repository - annotated tag ctdb-1.43 created - ctdb-1.43

2012-05-17 Thread Ronnie Sahlberg
The annotated tag, ctdb-1.43 has been created
at  35edf2005e525a9dd3e5d8c68b8737adbb7deb22 (tag)
   tagging  f66d2b30eedc755cb8bc8b1c1e18141c6ee0c946 (commit)
  replaces  ctdb-1.42
 tagged by  Ronnie Sahlberg
on  Thu May 17 18:40:31 2012 +1000

- Log -
Tag for 1.43
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)

iQEcBAABAgAGBQJPtLmYAAoJEB6eS+vwPaeGDNAH/il4ZWDeGzf1P8i/1IoCMbW6
W4XnESLoqckXtCTZyOK/5Yhylg/Abow8ToBfY6Wg1CepFb51GI+1HDQ4/bPY9fG9
OMWgXAGWyDHnEAe8BMqrYzOuj0BISvjLuwcu93cBeyQ7QHyNAHNNHWqQOK9uYYaF
4Ldi1o72aAJqVs+aFf4r4RMhITncJsLBFl69qc46IZW6O2e5sY1XB6eKVT0XNNqS
5yV5euIWuAOEusfGCD30DeGJxUUE7XgNipStdgbslHMaTTJEdxdmO7JRsCP7dwh4
bBi8IeCsjkw4MCR/KnjV8PS5/y3+1bw1TWXm1Ssbo8SwLXcqJOiP5MOMVDx6qGY=
=Lu8n
-END PGP SIGNATURE-

Amitay Isaacs (21):
  tests/tool: Fix the nodestatus test
  tests/tool: New nodestatus test
  tests: exportfs always outputs with options in brackets
  tests: Add a script to run cluster tests and make target test_cluster
  tests: Add regular expression parsing for hop_count_buckets
  tests: Fix the error messages in test event script
  ctdbd: Fix the error message string
  tests: Check for assigned IP addresses only if we are on real cluster
  tests: Check assigned IPs from ctdb output
  tests: Set the debug level = 3 when running local tests
  tests: Use CTDB_TEST_REAL_CLUSTER to decide if tests use local daemons
  recoverd: Fix spurious warnings when running with --nopublicipcheck
  ctdbd: Fix spurious warnings when running with --nopublicipcheck
  tests/tool: Fix the nodestatus test
  tests/tool: New nodestatus test
  tests: Fix wrapper scripts
  tests: CTDB_TEST_WRAPPER has to be an absolute path on a real cluster
  tests: test_wrap needs to set TEST_SCRIPTS_DIR
  tests/simple: Fix typo in the test message
  server: locking: Provide a common API for non-blocking locking of TDBs
  Revert server: locking: Provide a common API for non-blocking locking of 
TDBs

Martin Schwenke (72):
  Tests - turn of time logging by default
  Tests - IP allocation - add another LCP2 test based on a test failure
  Tests - IP allocation - 2 tests for NODE_FLAGS_NOIPTAKEOVER
  tests - export new variable TEST_SCRIPTS_DIR
  tests - add scripts/common.sh
  tests - add scripts/unit.sh
  tests - move functions only used by scripts/run_tests into that script
  tests - add -v option to set TEST_VERBOSE=true
  tests - run_tests can take a directory as an argument
  tests - run_tests includes common.sh, uses die()
  tests/tool - Restructure according to new convention
  tests/onnode - Restructure according to new convention
  tests/takeover - Restructure according to new convention
  tests - run_tests ignores trailing '/' on directories
  tests/eventscripts - Restructure according to new convention
  tests - run_tests needs to expand directories like . and ...
  tests/eventscripts: Tweak an error message in a policy routing test
  tests/eventscripts: Share directories must be absolute in eventscript 
tests
  tests: More unit test factoring/rationalisation and bug fixes
  tests: Rename ctdb_test_functions.bash to integration.bash
  tests: Rationalise integration test infrastructure
  Eventscripts - Fix typo in 13.per_ip_routing support for 
__auto_link_local__
  tests: Local daemons are no longer the default, now require run_tests -l
  tests: Move relative directory path hack from run_tests to common.sh
  tests: Programs run by tests should be found in $PATH
  tests: Remove ctdb_test_env
  tests: Add -e option to cause run_tests to exit on first test failure
  tests: Add new -H option for run_tests to avoid printing header/footer
  tests: Fix trailing whitespace issues in integration.bash
  tests: test_wrap can use $TEST_SCRIPTS_DIR for a little extra clarity
  tests: run_tests should exit with failed test status if running with -H
  tests: Update top-level wrapper scripts
  tests: Update README files and add new README files where missing
  tests: Make run_tests -X more flexible - it now works with onnode
  tests/simple: Replace tdbdump with ctdb cattdb
  tests/integration: Use absolute path for socket symlink
  tests/integration: Remove cabability for testcase option
  tests: Add a -V option to set new variable TEST_VAR_DIR
  tests: Update integration.sh to use TEST_VAR_DIR
  tests: Improve the logic for adding directories to $PATH
  tests: Unit tests should use $TEST_VAR_DIR
  tests: Time logging in integration tests should use TEST_VAR_DIR
  tests/eventscripts: $FAKE_IP_STATE is always set in ip command stub
  tests: New run_tests -C (cleanup) option
  tests/simple: Add new pseudo testcase to potentially shutdown local 
daemons
  tests: Add

[SCM] CTDB repository - branch 1.13 updated - ctdb-1.43

2012-05-17 Thread Ronnie Sahlberg
The branch, 1.13 has been updated
   via  f66d2b30eedc755cb8bc8b1c1e18141c6ee0c946 (commit)
  from  93b176b39dddc71011ec7182deda435874474e45 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=1.13


- Log -
commit f66d2b30eedc755cb8bc8b1c1e18141c6ee0c946
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Thu May 17 18:39:52 2012 +1000

New version 1.43

---

Summary of changes:
 packaging/RPM/ctdb.spec.in |4 
 1 files changed, 4 insertions(+), 0 deletions(-)


Changeset truncated at 500 lines:

diff --git a/packaging/RPM/ctdb.spec.in b/packaging/RPM/ctdb.spec.in
index 0367556..6d5b4a4 100644
--- a/packaging/RPM/ctdb.spec.in
+++ b/packaging/RPM/ctdb.spec.in
@@ -198,6 +198,10 @@ test suite for ctdb
 
 
 %changelog
+* Thu May 17 2012 : Version 1.32
+ - Update Ganesha script
+ - Add new script to collect data when eventscripts hang
+ - Tests updates
 * Thu May 3 2012 : Version 1.42
  - Add a new command 'reloadips' that can be used to re-read and update from
the public addresses file at runtime.


-- 
CTDB repository


[SCM] CTDB repository - branch master updated - ctdb-1.13-173-g6e68797

2012-05-16 Thread Ronnie Sahlberg
The branch, master has been updated
   via  6e68797af67bee36f2bad045f94806e7e98f27e9 (commit)
   via  2bcd58b30d7cf6dd48ad7f019810c6965a44c85a (commit)
   via  863ad337fa3c4effe1fd370d3ba414027c600bd6 (commit)
   via  f4b7d14f2e3c7345e7a09abb27c32923fb78cbc4 (commit)
   via  c7d6e4557d00de674737e2c8d6cbebaa2461c303 (commit)
   via  d661f09c6c76543bd091d51bc0e6fe97ed0f55d1 (commit)
   via  ed2db1f4e8d2b222d7f912a4a007ce48a23e83b0 (commit)
   via  d2514051761ba23deee0e118c25afab2ab8d854e (commit)
   via  a00e80c701a0f9695f41c24e0360c25c0873d49d (commit)
   via  6b8507d4d3062e709409b3790117d87311b3460d (commit)
   via  db8cf8f5e644a0b21a6040287887fee40f38d4db (commit)
   via  9f20fbf91706db94f65f62dbd6a4e087890c1da9 (commit)
   via  61c80f58a8cfbaca7e669ef8cd95b4f6b5dc66c7 (commit)
   via  e7ba0a30e7b7fe78e83e493128b349708cf0038a (commit)
   via  14a630cbc638d2f3bf699de5d94131e5f7d1a3ea (commit)
   via  1957d53b78f101cd0cd37d9705a225deef5174a2 (commit)
   via  f0a14169883661a702a352aa0311a2b433071a83 (commit)
   via  5c23244591b8005f91d8e25e719cd332c711db14 (commit)
  from  e3dc5bd3f1ef1f0ed08f57a5b5bafcac936e9ed0 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=master


- Log -
commit 6e68797af67bee36f2bad045f94806e7e98f27e9
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Thu May 17 10:17:51 2012 +1000

Debug: When scripts hang, we may need to collect additional data in order 
to debug why the script hung.

Break this debug and datacollection out into an external script to make it 
easier to modify what data we need to collect.
For now we only collect a pstree so we can see what part of the script we 
hung in.

S1037271

---

Summary of changes:
 Makefile.in |3 +-
 config/ctdb.init|1 +
 config/ctdb.sysconfig   |3 +
 config/debug-hung-script.sh |7 ++
 config/functions|   12 ++-
 include/ctdb_private.h  |3 +
 packaging/RPM/ctdb.spec.in  |   65 +++--
 server/ctdb_tunables.c  |2 +-
 server/ctdbd.c  |   11 +++
 server/eventscript.c|   66 -
 tests/INSTALL   |   10 +--
 tests/eventscripts/13.per_ip_routing.002.sh |2 +-
 tests/eventscripts/scripts/local.sh |3 +-
 tests/run_cluster_tests.sh  |   19 +-
 tests/run_tests.sh  |   40 --
 tests/scripts/integration.bash  |   42 --
 tests/scripts/run_tests |   60 +++-
 tests/simple/18_ctdb_reloadips.sh   |  106 +++
 18 files changed, 362 insertions(+), 93 deletions(-)
 create mode 100644 config/debug-hung-script.sh
 mode change 100755 = 12 tests/run_cluster_tests.sh
 create mode 100755 tests/simple/18_ctdb_reloadips.sh


Changeset truncated at 500 lines:

diff --git a/Makefile.in b/Makefile.in
index ecf03f2..cdebbd7 100755
--- a/Makefile.in
+++ b/Makefile.in
@@ -356,6 +356,7 @@ install: all $(PMDA_INSTALL)
if [ -f doc/ltdbtool.1 ]; then ${INSTALLCMD} -m 644 doc/ltdbtool.1 
$(DESTDIR)$(mandir)/man1; fi
if [ -f doc/ping_pong.1 ];then ${INSTALLCMD} -m 644 doc/ping_pong.1 
$(DESTDIR)$(mandir)/man1; fi
if [ ! -f $(DESTDIR)$(etcdir)/ctdb/notify.sh ];then ${INSTALLCMD} -m 
755 config/notify.sh $(DESTDIR)$(etcdir)/ctdb; fi
+   ${INSTALLCMD} -m 755 config/debug-hung-script.sh 
$(DESTDIR)$(etcdir)/ctdb
if [ ! -f $(DESTDIR)$(etcdir)/ctdb/ctdb-crash-cleanup.sh ];then 
${INSTALLCMD} -m 755 config/ctdb-crash-cleanup.sh $(DESTDIR)$(etcdir)/ctdb; fi
 
 install_pmda:
@@ -369,7 +370,7 @@ install_tests: all
tests/INSTALL --destdir=$(DESTDIR) --datarootdir=$(prefix)/share 
--libdir=$(libdir) --bindir=$(bindir) --etcdir=$(etcdir)
 
 test: all
-   tests/run_tests.sh
+   tests/run_tests.sh -V tests/var
 
 test_cluster: all
tests/run_cluster_tests.sh
diff --git a/config/ctdb.init b/config/ctdb.init
index 0a4d29b..5d01a9c 100755
--- a/config/ctdb.init
+++ b/config/ctdb.init
@@ -95,6 +95,7 @@ build_ctdb_options () {
 maybe_set --event-script-dir   $CTDB_EVENT_SCRIPT_DIR
 maybe_set --transport  $CTDB_TRANSPORT
 maybe_set -d   $CTDB_DEBUGLEVEL
+maybe_set --debug-hung-script  $CTDB_DEBUG_HUNG_SCRIPT
 maybe_set --notification-script$CTDB_NOTIFY_SCRIPT
 maybe_set --start-as-disabled  $CTDB_START_AS_DISABLEDyes
 maybe_set --start-as-stopped   $CTDB_START_AS_STOPPED yes
diff --git a/config/ctdb.sysconfig b/config/ctdb.sysconfig
index 163a0dd..fecc7a5

[SCM] CTDB repository - branch 1.13 updated - ctdb-1.42-108-g93b176b

2012-05-16 Thread Ronnie Sahlberg
The branch, 1.13 has been updated
   via  93b176b39dddc71011ec7182deda435874474e45 (commit)
   via  06041c372525924add825bf23a0758546465a516 (commit)
   via  6f30c8d233b150d12f7e37e54769fdd670437b90 (commit)
  from  6ebd66a10b3fce30cf686f67349c4372c1042dc9 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=1.13


- Log -
commit 93b176b39dddc71011ec7182deda435874474e45
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Thu May 17 11:16:57 2012 +1000

GANESHA: make the ganesha script executable by default

commit 06041c372525924add825bf23a0758546465a516
Author: Martin Schwenke mar...@meltin.net
Date:   Wed May 16 17:24:21 2012 +1000

Eventscripts: Modernise 60.ganesha to match 60.nfs

Originally from Srikrishan Malik srikrishan.ma...@in.ibm.com with
some style changes by me.

Signed-off-by: Martin Schwenke mar...@meltin.net

commit 6f30c8d233b150d12f7e37e54769fdd670437b90
Author: Martin Schwenke mar...@meltin.net
Date:   Wed May 16 13:29:58 2012 +1000

Eventscripts: restart lockd in the background when going unhealthy

Sometimes the restart can hang when there are I/O problems.  Then the
eventscript times out and gets killed so the node never marked as
unhealthy.

Restarting in the background avoids this.

Signed-off-by: Martin Schwenke mar...@meltin.net

---

Summary of changes:
 Makefile.in|2 +-
 config/events.d/60.ganesha |  207 
 config/events.d/60.nfs |4 +-
 config/functions   |   31 +++
 config/statd-callout   |   14 ++-
 5 files changed, 137 insertions(+), 121 deletions(-)


Changeset truncated at 500 lines:

diff --git a/Makefile.in b/Makefile.in
index cdebbd7..cbb987a 100755
--- a/Makefile.in
+++ b/Makefile.in
@@ -343,7 +343,7 @@ install: all $(PMDA_INSTALL)
${INSTALLCMD} -m 755 config/events.d/41.httpd 
$(DESTDIR)$(etcdir)/ctdb/events.d
${INSTALLCMD} -m 755 config/events.d/50.samba 
$(DESTDIR)$(etcdir)/ctdb/events.d
${INSTALLCMD} -m 755 config/events.d/60.nfs 
$(DESTDIR)$(etcdir)/ctdb/events.d
-   ${INSTALLCMD} -m 644 config/events.d/60.ganesha 
$(DESTDIR)$(etcdir)/ctdb/events.d
+   ${INSTALLCMD} -m 755 config/events.d/60.ganesha 
$(DESTDIR)$(etcdir)/ctdb/events.d
${INSTALLCMD} -m 755 config/events.d/62.cnfs 
$(DESTDIR)$(etcdir)/ctdb/events.d
${INSTALLCMD} -m 755 config/events.d/70.iscsi 
$(DESTDIR)$(etcdir)/ctdb/events.d
${INSTALLCMD} -m 755 config/events.d/91.lvs 
$(DESTDIR)$(etcdir)/ctdb/events.d
diff --git a/config/events.d/60.ganesha b/config/events.d/60.ganesha
index fb3b7c2..cee7792 100755
--- a/config/events.d/60.ganesha
+++ b/config/events.d/60.ganesha
@@ -1,34 +1,58 @@
 #!/bin/sh
 # script to manage nfs in a clustered environment
 
-start_nfs() {
-   mkdir -p $CTDB_VARDIR/state/nfs
-   mkdir -p $CTDB_VARDIR/state/statd/ip
-   ctdb_service_stop
-   ctdb_service_start
-   echo 1  /proc/sys/net/ipv4/tcp_tw_recycle
+. $CTDB_BASE/functions
+
+service_name=nfs-ganesha-gpfs
+
+
+service_start ()
+{
+startstop_ganesha stop
+startstop_ganesha start
+set_proc sys/net/ipv4/tcp_tw_recycle 1
 }
 
-. $CTDB_BASE/functions
+service_stop ()
+{
+startstop_ganesha stop
+}
+
+service_reconfigure ()
+{
+# if the ips have been reallocated, we must restart ganesha
+# across all nodes and ping all statd listeners
+[ -x $CTDB_BASE/statd-callout ]  {
+   $CTDB_BASE/statd-callout notify 
+} /dev/null 21
+}
+loadconfig nfs
 
-loadconfig nfs
+[ $NFS_SERVER_MODE == ganesha ] || exit 0
 
-[ $NFS_SERVER_MODE = GANESHA ] || exit 0
+ctdb_setup_service_state_dir
+
+statd_update_trigger=$service_state_dir/update-trigger
+# We want this file to always exist.  The corner case is when
+# auto-start/stop is switched off, NFS is added as a managed service
+# some time after ctdbd is started and someone else starts the NFS
+# service for us.  In this case this file might not otherwise exist
+# when we get to a monitor event.
+touch $statd_update_trigger
 
-service_name=nfs-ganesha-gpfs
 
 ctdb_start_stop_service
 
 is_ctdb_managed_service || exit 0
 
+ctdb_service_check_reconfigure
+
 case $1 in
  init)
# read statd from persistent database
;;
  startup)
ctdb_service_start
-   mkdir -p $CTDB_VARDIR/state/statd
-   touch $CTDB_VARDIR/state/statd/update-trigger
;;
 
  shutdown)
@@ -44,111 +68,68 @@ case $1 in
;;
 
   monitor)
-   if ctdb_service_needs_reconfigure ; then
-   ctdb_service_reconfigure
-   exit 0
-   fi
 
update_tickles 2049
-
-   # check that statd responds to rpc requests
-   # if statd is not running we try to restart it
-   if ctdb_check_rpc STATD status 1 /dev/null

[SCM] CTDB repository - branch master updated - ctdb-1.13-178-gf59b40b

2012-05-16 Thread Ronnie Sahlberg
The branch, master has been updated
   via  f59b40b3f8ea3da8ffb8601bc025e83c237072d5 (commit)
   via  f23b5a160184db8c92f8c69307dc4a64adae839d (commit)
   via  637cab6304dae66b85668506028c76ea1ee88980 (commit)
   via  13acd58c41fba1a33894fbd654fed69ea0eac322 (commit)
   via  92f74fd589467b46c758e116e97417edfe8773d7 (commit)
  from  6e68797af67bee36f2bad045f94806e7e98f27e9 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=master


- Log -
commit f59b40b3f8ea3da8ffb8601bc025e83c237072d5
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Thu May 17 11:16:57 2012 +1000

GANESHA: make the ganesha script executable by default

commit f23b5a160184db8c92f8c69307dc4a64adae839d
Merge: 6e68797af67bee36f2bad045f94806e7e98f27e9 
637cab6304dae66b85668506028c76ea1ee88980
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Thu May 17 11:48:07 2012 +1000

Merge remote branch 'martins/ganesha'

commit 637cab6304dae66b85668506028c76ea1ee88980
Author: Martin Schwenke mar...@meltin.net
Date:   Wed May 16 17:24:21 2012 +1000

Eventscripts: Modernise 60.ganesha to match 60.nfs

Originally from Srikrishan Malik srikrishan.ma...@in.ibm.com with
some style changes by me.

Signed-off-by: Martin Schwenke mar...@meltin.net

commit 13acd58c41fba1a33894fbd654fed69ea0eac322
Author: Martin Schwenke mar...@meltin.net
Date:   Wed May 16 13:29:58 2012 +1000

Eventscripts: restart lockd in the background when going unhealthy

Sometimes the restart can hang when there are I/O problems.  Then the
eventscript times out and gets killed so the node never marked as
unhealthy.

Restarting in the background avoids this.

Signed-off-by: Martin Schwenke mar...@meltin.net

commit 92f74fd589467b46c758e116e97417edfe8773d7
Author: Martin Schwenke mar...@meltin.net
Date:   Tue May 8 14:53:58 2012 +1000

Eventscript functions: add optional version to nfs_check_rpc_service()

This can be optional because the 1st item of each action-triple is a
test comparison that starts with '-'.

Signed-off-by: Martin Schwenke mar...@meltin.net

---

Summary of changes:
 Makefile.in|2 +-
 config/events.d/60.ganesha |  207 
 config/events.d/60.nfs |4 +-
 config/functions   |   31 +++
 config/statd-callout   |   14 ++-
 5 files changed, 137 insertions(+), 121 deletions(-)


Changeset truncated at 500 lines:

diff --git a/Makefile.in b/Makefile.in
index cdebbd7..cbb987a 100755
--- a/Makefile.in
+++ b/Makefile.in
@@ -343,7 +343,7 @@ install: all $(PMDA_INSTALL)
${INSTALLCMD} -m 755 config/events.d/41.httpd 
$(DESTDIR)$(etcdir)/ctdb/events.d
${INSTALLCMD} -m 755 config/events.d/50.samba 
$(DESTDIR)$(etcdir)/ctdb/events.d
${INSTALLCMD} -m 755 config/events.d/60.nfs 
$(DESTDIR)$(etcdir)/ctdb/events.d
-   ${INSTALLCMD} -m 644 config/events.d/60.ganesha 
$(DESTDIR)$(etcdir)/ctdb/events.d
+   ${INSTALLCMD} -m 755 config/events.d/60.ganesha 
$(DESTDIR)$(etcdir)/ctdb/events.d
${INSTALLCMD} -m 755 config/events.d/62.cnfs 
$(DESTDIR)$(etcdir)/ctdb/events.d
${INSTALLCMD} -m 755 config/events.d/70.iscsi 
$(DESTDIR)$(etcdir)/ctdb/events.d
${INSTALLCMD} -m 755 config/events.d/91.lvs 
$(DESTDIR)$(etcdir)/ctdb/events.d
diff --git a/config/events.d/60.ganesha b/config/events.d/60.ganesha
index fb3b7c2..cee7792 100755
--- a/config/events.d/60.ganesha
+++ b/config/events.d/60.ganesha
@@ -1,34 +1,58 @@
 #!/bin/sh
 # script to manage nfs in a clustered environment
 
-start_nfs() {
-   mkdir -p $CTDB_VARDIR/state/nfs
-   mkdir -p $CTDB_VARDIR/state/statd/ip
-   ctdb_service_stop
-   ctdb_service_start
-   echo 1  /proc/sys/net/ipv4/tcp_tw_recycle
+. $CTDB_BASE/functions
+
+service_name=nfs-ganesha-gpfs
+
+
+service_start ()
+{
+startstop_ganesha stop
+startstop_ganesha start
+set_proc sys/net/ipv4/tcp_tw_recycle 1
 }
 
-. $CTDB_BASE/functions
+service_stop ()
+{
+startstop_ganesha stop
+}
+
+service_reconfigure ()
+{
+# if the ips have been reallocated, we must restart ganesha
+# across all nodes and ping all statd listeners
+[ -x $CTDB_BASE/statd-callout ]  {
+   $CTDB_BASE/statd-callout notify 
+} /dev/null 21
+}
+loadconfig nfs
 
-loadconfig nfs
+[ $NFS_SERVER_MODE == ganesha ] || exit 0
 
-[ $NFS_SERVER_MODE = GANESHA ] || exit 0
+ctdb_setup_service_state_dir
+
+statd_update_trigger=$service_state_dir/update-trigger
+# We want this file to always exist.  The corner case is when
+# auto-start/stop is switched off, NFS is added as a managed service
+# some time after ctdbd is started and someone else starts the NFS
+# service for us.  In this case this file might not otherwise exist
+# when we get to a monitor event.
+touch

[SCM] CTDB repository - branch master updated - ctdb-1.13-155-ge3dc5bd

2012-05-10 Thread Ronnie Sahlberg
The branch, master has been updated
   via  e3dc5bd3f1ef1f0ed08f57a5b5bafcac936e9ed0 (commit)
  from  f73a4b1495830bcdd094a93732a89dd53b3c2f78 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=master


- Log -
commit e3dc5bd3f1ef1f0ed08f57a5b5bafcac936e9ed0
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Thu May 10 17:40:22 2012 +1000

We dont need to serialize the probe which address this node is if we have 
given an explicit --node-ip on the commandline

---

Summary of changes:
 tcp/tcp_connect.c |   47 +++
 1 files changed, 27 insertions(+), 20 deletions(-)


Changeset truncated at 500 lines:

diff --git a/tcp/tcp_connect.c b/tcp/tcp_connect.c
index 0e1318f..2814201 100644
--- a/tcp/tcp_connect.c
+++ b/tcp/tcp_connect.c
@@ -281,26 +281,29 @@ static int ctdb_tcp_listen_automatic(struct ctdb_context 
*ctdb)
int sock_size;
struct tevent_fd *fde;
 
-   /* in order to ensure that we don't get two nodes with the
-  same adddress, we must make the bind() and listen() calls
-  atomic. The SO_REUSEADDR setsockopt only prevents double
-  binds if the first socket is in LISTEN state  */
-   lock_fd = open(lock_path, O_RDWR|O_CREAT, 0666);
-   if (lock_fd == -1) {
-   DEBUG(DEBUG_CRIT,(Unable to open %s\n, lock_path));
-   return -1;
-   }
+   /* We only need to serialize this if we dont yet know the node ip */
+   if (!ctdb-node_ip) {
+   /* in order to ensure that we don't get two nodes with the
+  same adddress, we must make the bind() and listen() calls
+  atomic. The SO_REUSEADDR setsockopt only prevents double
+  binds if the first socket is in LISTEN state  */
+   lock_fd = open(lock_path, O_RDWR|O_CREAT, 0666);
+   if (lock_fd == -1) {
+   DEBUG(DEBUG_CRIT,(Unable to open %s\n, lock_path));
+   return -1;
+   }
 
-   lock.l_type = F_WRLCK;
-   lock.l_whence = SEEK_SET;
-   lock.l_start = 0;
-   lock.l_len = 1;
-   lock.l_pid = 0;
+   lock.l_type = F_WRLCK;
+   lock.l_whence = SEEK_SET;
+   lock.l_start = 0;
+   lock.l_len = 1;
+   lock.l_pid = 0;
 
-   if (fcntl(lock_fd, F_SETLKW, lock) != 0) {
-   DEBUG(DEBUG_CRIT,(Unable to lock %s\n, lock_path));
-   close(lock_fd);
-   return -1;
+   if (fcntl(lock_fd, F_SETLKW, lock) != 0) {
+   DEBUG(DEBUG_CRIT,(Unable to lock %s\n, lock_path));
+   close(lock_fd);
+   return -1;
+   }
}
 
for (i=0; i  ctdb-num_nodes; i++) {
@@ -399,11 +402,15 @@ static int ctdb_tcp_listen_automatic(struct ctdb_context 
*ctdb)
   ctdb_listen_event, ctdb);
tevent_fd_set_auto_close(fde);
 
-   close(lock_fd);
+   if (!ctdb-node_ip) {
+   close(lock_fd);
+   }
return 0;

 failed:
-   close(lock_fd);
+   if (!ctdb-node_ip) {
+   close(lock_fd);
+   }
close(ctcp-listen_fd);
ctcp-listen_fd = -1;
return -1;


-- 
CTDB repository


[SCM] CTDB repository - branch 1.13 updated - ctdb-1.42-2-g176f6aa

2012-05-10 Thread Ronnie Sahlberg
The branch, 1.13 has been updated
   via  176f6aae0f5ec276ecc35f4bbe633d37af6bc035 (commit)
  from  b550f4644ea81b7010d401e46cf803b13f16e3c1 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=1.13


- Log -
commit 176f6aae0f5ec276ecc35f4bbe633d37af6bc035
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Thu May 10 17:40:22 2012 +1000

We dont need to serialize the probe which address this node is if we have 
given an explicit --node-ip on the commandline

---

Summary of changes:
 tcp/tcp_connect.c |   47 +++
 1 files changed, 27 insertions(+), 20 deletions(-)


Changeset truncated at 500 lines:

diff --git a/tcp/tcp_connect.c b/tcp/tcp_connect.c
index 0e1318f..2814201 100644
--- a/tcp/tcp_connect.c
+++ b/tcp/tcp_connect.c
@@ -281,26 +281,29 @@ static int ctdb_tcp_listen_automatic(struct ctdb_context 
*ctdb)
int sock_size;
struct tevent_fd *fde;
 
-   /* in order to ensure that we don't get two nodes with the
-  same adddress, we must make the bind() and listen() calls
-  atomic. The SO_REUSEADDR setsockopt only prevents double
-  binds if the first socket is in LISTEN state  */
-   lock_fd = open(lock_path, O_RDWR|O_CREAT, 0666);
-   if (lock_fd == -1) {
-   DEBUG(DEBUG_CRIT,(Unable to open %s\n, lock_path));
-   return -1;
-   }
+   /* We only need to serialize this if we dont yet know the node ip */
+   if (!ctdb-node_ip) {
+   /* in order to ensure that we don't get two nodes with the
+  same adddress, we must make the bind() and listen() calls
+  atomic. The SO_REUSEADDR setsockopt only prevents double
+  binds if the first socket is in LISTEN state  */
+   lock_fd = open(lock_path, O_RDWR|O_CREAT, 0666);
+   if (lock_fd == -1) {
+   DEBUG(DEBUG_CRIT,(Unable to open %s\n, lock_path));
+   return -1;
+   }
 
-   lock.l_type = F_WRLCK;
-   lock.l_whence = SEEK_SET;
-   lock.l_start = 0;
-   lock.l_len = 1;
-   lock.l_pid = 0;
+   lock.l_type = F_WRLCK;
+   lock.l_whence = SEEK_SET;
+   lock.l_start = 0;
+   lock.l_len = 1;
+   lock.l_pid = 0;
 
-   if (fcntl(lock_fd, F_SETLKW, lock) != 0) {
-   DEBUG(DEBUG_CRIT,(Unable to lock %s\n, lock_path));
-   close(lock_fd);
-   return -1;
+   if (fcntl(lock_fd, F_SETLKW, lock) != 0) {
+   DEBUG(DEBUG_CRIT,(Unable to lock %s\n, lock_path));
+   close(lock_fd);
+   return -1;
+   }
}
 
for (i=0; i  ctdb-num_nodes; i++) {
@@ -399,11 +402,15 @@ static int ctdb_tcp_listen_automatic(struct ctdb_context 
*ctdb)
   ctdb_listen_event, ctdb);
tevent_fd_set_auto_close(fde);
 
-   close(lock_fd);
+   if (!ctdb-node_ip) {
+   close(lock_fd);
+   }
return 0;

 failed:
-   close(lock_fd);
+   if (!ctdb-node_ip) {
+   close(lock_fd);
+   }
close(ctcp-listen_fd);
ctcp-listen_fd = -1;
return -1;


-- 
CTDB repository


[SCM] CTDB repository - branch 1.13 updated - ctdb-1.13-104-g16282b5

2012-05-02 Thread Ronnie Sahlberg
The branch, 1.13 has been updated
   via  16282b5b2b8fb14454ab61a7ff7c651528a4a117 (commit)
   via  e1c47aec5b0428d354d8b911d895b77668a99507 (commit)
  from  d697e75605e94bfeb57854995b9af2a558849d56 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=1.13


- Log -
commit 16282b5b2b8fb14454ab61a7ff7c651528a4a117
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Thu May 3 11:09:01 2012 +1000

New version 1.42

commit e1c47aec5b0428d354d8b911d895b77668a99507
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Thu May 3 11:06:55 2012 +1000

DOC: document the reloadips command

---

Summary of changes:
 doc/ctdb.1 |   22 +-
 doc/ctdb.1.html|  183 +---
 doc/ctdb.1.xml |   44 +++
 packaging/RPM/ctdb.spec.in |5 +
 4 files changed, 175 insertions(+), 79 deletions(-)


Changeset truncated at 500 lines:

diff --git a/doc/ctdb.1 b/doc/ctdb.1
index c75a658..2a0ec62 100644
--- a/doc/ctdb.1
+++ b/doc/ctdb.1
@@ -2,7 +2,7 @@
 .\ It was generated using the DocBook XSL Stylesheets (version 1.69.1).
 .\ Instead of manually editing it, you probably should edit the DocBook XML
 .\ source for it and then use the DocBook XSL Stylesheets to regenerate it.
-.TH CTDB 1 02/13/2012 ctdb CTDB \- clustered TDB database
+.TH CTDB 1 05/03/2012 ctdb CTDB \- clustered TDB database
 .\ disable hyphenation
 .nh
 .\ disable justification (adjust text to left margin only)
@@ -619,6 +619,26 @@ Procedure to remove a node:
 .PP
 5, Use 'ctdb status' on all nodes and verify that the deleted node no longer 
shows up in the list..
 .PP
+.SS reloadips
+.PP
+This command is used to reload the public addresses file and update the ip 
configuration of the running daemon.
+.PP
+Procedure to update the public address configuration on a single node:
+.PP
+1, Update the /etc/ctdb/public_addresses file on the node
+.PP
+2, Run 'ctdb reloadips' on the node.
+.PP
+The file will then be reloaded on the node and addresses will be added or 
removed as required to match the newly loaded file. When updating a single node 
it may take a little while before any newly added addresses are failed onto the 
node.
+.PP
+.PP
+Procedure to update the public address configuration on whole cluster:
+.PP
+1, Update the /etc/ctdb/public_addresses file on all nodes
+.PP
+2, Run 'ctdb reloadips \-n all'.
+.PP
+This command will then force all nodes to reload and update the addresses. 
This process is controlled and synchronized by the recovery master to ensure 
that all addresses are added to all nodes as one single operation, after which 
any required ip node rebalancing may may take place.
 .SS tickle srcip:port dstip:port
 .PP
 This command will will send a TCP tickle to the source host for the specified 
TCP connection. A TCP tickle is a TCP ACK packet with an invalid sequence and 
acknowledge number and will when received by the source host result in it 
sending an immediate correct ACK back to the other end.
diff --git a/doc/ctdb.1.html b/doc/ctdb.1.html
index a6da3fb..2b81389 100644
--- a/doc/ctdb.1.html
+++ b/doc/ctdb.1.html
@@ -1,6 +1,6 @@
-htmlheadmeta http-equiv=Content-Type content=text/html; 
charset=ISO-8859-1titlectdb/titlemeta name=generator content=DocBook 
XSL Stylesheets V1.69.1/headbody bgcolor=white text=black 
link=#FF vlink=#840084 alink=#FFdiv class=refentry 
lang=ena name=ctdb.1/adiv class=titlepage/divdiv 
class=refnamedivh2Name/h2pctdb #8212; clustered tdb database 
management utility/p/divdiv class=refsynopsisdivh2Synopsis/h2div 
class=cmdsynopsispcode class=commandctdb [ OPTIONS ] COMMAND 
.../code /p/divdiv class=cmdsynopsispcode 
class=commandctdb/code  [-n lt;nodegt;] [-Y] [-t lt;timeoutgt;] [-T 
lt;timelimitgt;] [-? --help] [--usage] [-d --debug=lt;INTEGERgt;] 
[--socket=lt;filenamegt;] [--print-emptyrecords] [--print-datasize] 
[--print-lmaster] [--print-hash] [--print-recordflags]/p/div/divdiv 
class=refsect1 lang=ena name=id4550272/ah2DESCRIPTION/h2p
+htmlheadmeta http-equiv=Content-Type content=text/html; 
charset=ISO-8859-1titlectdb/titlemeta name=generator content=DocBook 
XSL Stylesheets V1.69.1/headbody bgcolor=white text=black 
link=#FF vlink=#840084 alink=#FFdiv class=refentry 
lang=ena name=ctdb.1/adiv class=titlepage/divdiv 
class=refnamedivh2Name/h2pctdb #8212; clustered tdb database 
management utility/p/divdiv class=refsynopsisdivh2Synopsis/h2div 
class=cmdsynopsispcode class=commandctdb [ OPTIONS ] COMMAND 
.../code /p/divdiv class=cmdsynopsispcode 
class=commandctdb/code  [-n lt;nodegt;] [-Y] [-t lt;timeoutgt;] [-T 
lt;timelimitgt;] [-? --help] [--usage] [-d --debug=lt;INTEGERgt;] 
[--socket=lt;filenamegt;] [--print-emptyrecords] [--print-datasize] 
[--print-lmaster] [--print-hash] [--print-recordflags]/p/div/divdiv 
class=refsect1 lang=ena name

[SCM] CTDB repository - branch master updated - ctdb-1.13-153-g90e8c4d

2012-05-02 Thread Ronnie Sahlberg
The branch, master has been updated
   via  90e8c4dc7d89c460ad2da5a9af1cef006db3b1c0 (commit)
  from  334ea830f1bf33419f4a1e78f23afd41a852d0f4 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=master


- Log -
commit 90e8c4dc7d89c460ad2da5a9af1cef006db3b1c0
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Thu May 3 11:06:55 2012 +1000

DOC: document the reloadips command

---

Summary of changes:
 doc/ctdb.1  |   22 ++-
 doc/ctdb.1.html |  183 +++---
 doc/ctdb.1.xml  |   44 +
 3 files changed, 170 insertions(+), 79 deletions(-)


Changeset truncated at 500 lines:

diff --git a/doc/ctdb.1 b/doc/ctdb.1
index c75a658..2a0ec62 100644
--- a/doc/ctdb.1
+++ b/doc/ctdb.1
@@ -2,7 +2,7 @@
 .\ It was generated using the DocBook XSL Stylesheets (version 1.69.1).
 .\ Instead of manually editing it, you probably should edit the DocBook XML
 .\ source for it and then use the DocBook XSL Stylesheets to regenerate it.
-.TH CTDB 1 02/13/2012 ctdb CTDB \- clustered TDB database
+.TH CTDB 1 05/03/2012 ctdb CTDB \- clustered TDB database
 .\ disable hyphenation
 .nh
 .\ disable justification (adjust text to left margin only)
@@ -619,6 +619,26 @@ Procedure to remove a node:
 .PP
 5, Use 'ctdb status' on all nodes and verify that the deleted node no longer 
shows up in the list..
 .PP
+.SS reloadips
+.PP
+This command is used to reload the public addresses file and update the ip 
configuration of the running daemon.
+.PP
+Procedure to update the public address configuration on a single node:
+.PP
+1, Update the /etc/ctdb/public_addresses file on the node
+.PP
+2, Run 'ctdb reloadips' on the node.
+.PP
+The file will then be reloaded on the node and addresses will be added or 
removed as required to match the newly loaded file. When updating a single node 
it may take a little while before any newly added addresses are failed onto the 
node.
+.PP
+.PP
+Procedure to update the public address configuration on whole cluster:
+.PP
+1, Update the /etc/ctdb/public_addresses file on all nodes
+.PP
+2, Run 'ctdb reloadips \-n all'.
+.PP
+This command will then force all nodes to reload and update the addresses. 
This process is controlled and synchronized by the recovery master to ensure 
that all addresses are added to all nodes as one single operation, after which 
any required ip node rebalancing may may take place.
 .SS tickle srcip:port dstip:port
 .PP
 This command will will send a TCP tickle to the source host for the specified 
TCP connection. A TCP tickle is a TCP ACK packet with an invalid sequence and 
acknowledge number and will when received by the source host result in it 
sending an immediate correct ACK back to the other end.
diff --git a/doc/ctdb.1.html b/doc/ctdb.1.html
index a6da3fb..2b81389 100644
--- a/doc/ctdb.1.html
+++ b/doc/ctdb.1.html
@@ -1,6 +1,6 @@
-htmlheadmeta http-equiv=Content-Type content=text/html; 
charset=ISO-8859-1titlectdb/titlemeta name=generator content=DocBook 
XSL Stylesheets V1.69.1/headbody bgcolor=white text=black 
link=#FF vlink=#840084 alink=#FFdiv class=refentry 
lang=ena name=ctdb.1/adiv class=titlepage/divdiv 
class=refnamedivh2Name/h2pctdb #8212; clustered tdb database 
management utility/p/divdiv class=refsynopsisdivh2Synopsis/h2div 
class=cmdsynopsispcode class=commandctdb [ OPTIONS ] COMMAND 
.../code /p/divdiv class=cmdsynopsispcode 
class=commandctdb/code  [-n lt;nodegt;] [-Y] [-t lt;timeoutgt;] [-T 
lt;timelimitgt;] [-? --help] [--usage] [-d --debug=lt;INTEGERgt;] 
[--socket=lt;filenamegt;] [--print-emptyrecords] [--print-datasize] 
[--print-lmaster] [--print-hash] [--print-recordflags]/p/div/divdiv 
class=refsect1 lang=ena name=id4550272/ah2DESCRIPTION/h2p
+htmlheadmeta http-equiv=Content-Type content=text/html; 
charset=ISO-8859-1titlectdb/titlemeta name=generator content=DocBook 
XSL Stylesheets V1.69.1/headbody bgcolor=white text=black 
link=#FF vlink=#840084 alink=#FFdiv class=refentry 
lang=ena name=ctdb.1/adiv class=titlepage/divdiv 
class=refnamedivh2Name/h2pctdb #8212; clustered tdb database 
management utility/p/divdiv class=refsynopsisdivh2Synopsis/h2div 
class=cmdsynopsispcode class=commandctdb [ OPTIONS ] COMMAND 
.../code /p/divdiv class=cmdsynopsispcode 
class=commandctdb/code  [-n lt;nodegt;] [-Y] [-t lt;timeoutgt;] [-T 
lt;timelimitgt;] [-? --help] [--usage] [-d --debug=lt;INTEGERgt;] 
[--socket=lt;filenamegt;] [--print-emptyrecords] [--print-datasize] 
[--print-lmaster] [--print-hash] [--print-recordflags]/p/div/divdiv 
class=refsect1 lang=ena name=id1890944/ah2DESCRIPTION/h2p
   ctdb is a utility to view and manage a ctdb cluster.
-/p/divdiv class=refsect1 lang=ena 
name=id4550281/ah2OPTIONS/h2div class=variablelistdldtspan 
class=term-n lt;pnngt;/span/dtddp
+/p/divdiv class=refsect1 lang=ena 
name=id1890953

[SCM] CTDB repository - annotated tag ctdb-1.13.1 created - ctdb-1.13.1

2012-05-02 Thread Ronnie Sahlberg
The annotated tag, ctdb-1.13.1 has been created
at  3214c89c5e5ac17ed897d66e247ca57f0ffc2730 (tag)
   tagging  980beee0aea319d8670a538858895ba662e036f8 (commit)
  replaces  ctdb-1.13
 tagged by  Ronnie Sahlberg
on  Wed Mar 21 16:41:47 2012 +1100

- Log -
Tag for 1.13.1
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)

iQEcBAABAgAGBQJPaWojAAoJEB6eS+vwPaeGVG4IALN8dn80rBjWe7seFe8WXzla
fMwDwrNMEQ6pL2A+qX4VW/p8La1P5sblP32nDJVPPHqzoj161fwFgyht0YBePfh0
P6b7cwtIR6d/JgStWk08JZB4tInvJyCDSEMGhhFow5Ou8oGEDelivk5zQojOeFAz
Va7pT1rfbMwxDvWqf9lj7CiRw9CLjgqF91JxcZA2PeehRv8/UFldZDxnWrzFePH1
sMaTNViFNUVTN8RHzmfHrO8wH/+ImLL4y03JaMzZwR5iIMyHbNuow3g4uT/DIDzc
z+aVNNHYNHUnVDYtQVjaYaieK1pl822WnNrZr2b9hId0xcxIbPcj5zf8E2FSM6g=
=oO/g
-END PGP SIGNATURE-

Ronnie Sahlberg (13):
  READONLY: when updating a remote node to revoke a delegation, make sure 
we dont create the record if it doesnt already exist
  ReadOnly: Only restrict the readonly sanity/paranoia check that only 
allows
  add max hop count buckets to see how bad hopcounts are
  FETCH COLLAPSE : Change the fetch-lock collapse to collapse ALL fetches, 
including fetch-locks into a single command in flight per record. Also add a 
tunable to enable/disable this optimization for hot records
  DOC document the FetchCollapse tunable
  CTDB: change how the hop count buckets are displayed in ctdb statistics
  STATISTICS: add per-db hop count statistics
  LACOUNT:  Add back lacount mechanism to defer migrating a fetched/read 
copy until after default of 20 consecutive requests from the same node
  DOC: Document the MaxLACount tunable variable
  STICKY: add prototype code to make records stick to a node to calm down 
if they are found to be very hot and accessed by a lot of clients.
  STICKY   document the sticky record parameters
  DOC  regenerate the manpage
  New version 1.13.1

---


-- 
CTDB repository


[SCM] CTDB repository - annotated tag ctdb-1.41 created - ctdb-1.41

2012-05-02 Thread Ronnie Sahlberg
The annotated tag, ctdb-1.41 has been created
at  fc0a526c6c29afd0ce67b32fa7e98d518a4c6282 (tag)
   tagging  0489d75a27b63f3f2e69f18400498e667995738a (commit)
  replaces  ctdb-1.40
 tagged by  Ronnie Sahlberg
on  Tue Apr 3 14:06:08 2012 +1000

- Log -
tag for 1.41
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)

iQEcBAABAgAGBQJPenc5AAoJEB6eS+vwPaeGae0IAO1UDYU8D7Ox2yn3kJrClx28
yEJIaxhwTdruOvN7v6UdZJTgsbsiKc2A7qcq6yu3jN6yR36L7TxnSoT9KSrZ2Ciq
K+l2QJ/qaEkKXdhh9hkOAyh0vR4aq/USEeQV8HzP4+ZF9/u89cvgisnsqZIb8VSs
AbdhCVumgiwen82MAeGkpUkvBXulJ4gyzd141hIFvQhPtyjAjDsrGcSpTYSVwBF8
LP4cl+4ywlJVcBsI2sThUGCAS/ac8noVjEX4bRvEHMMV2801ClrimNyoZ8xhPjhK
P0wVyndmBEyEK5yN3zRKmJBXCB3Jq3ZlqMEOrIjeli0URIKKnSsTe5VdmUzm+u0=
=qM5u
-END PGP SIGNATURE-

Martin Schwenke (1):
  Initscript - add backup of corrupt non-persistent databases

Ronnie Sahlberg (2):
  CTDB: add back the rebalanceip command
  New version 1.41

---


-- 
CTDB repository


[SCM] CTDB repository - annotated tag ctdb-1.40 created - ctdb-1.40

2012-05-02 Thread Ronnie Sahlberg
The annotated tag, ctdb-1.40 has been created
at  ed982a0e2aa418b77fc604cd36508331a1a03cfb (tag)
   tagging  c4aa79c2a89d362fe54c6bd86dc1831236047cfc (commit)
  replaces  ctdb-1.13.1
 tagged by  Ronnie Sahlberg
on  Mon Mar 26 06:12:49 2012 +1100

- Log -
Tag for 1.40
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)

iQEcBAABAgAGBQJPb246AAoJEB6eS+vwPaeG5C4IAOmTuuT+xTqKbgjC1tAlN03z
rE1du+61yFJpSstHk+KDiupyHlGgyWh58ij+dMHFl4D/flTyRSQD9fDQONw0Qh3z
/w6RUE4Xdn0wy+AyimTkyEu3V/WOaWPpUeT15s2bjwpB+eJJjtDfBjyIEnHnwIMw
QPGRKwJ47opj8ws5cZTi3gSimfOkeSf9c89TaQcA7vSGZpjdGCW2Yi3W+POtzDd2
XoKkoNL6+augiJoxLXtDUEWIVgEKy0UP2Ml9sBQNDxMwEgZODdylVjgtoCP25tKq
vNMRU97kmT2JDCkbhVuqpsd2y++BcMMNIxhODB1nQ7VZqLua5U1X6ok8Z2vgXu0=
=uSjV
-END PGP SIGNATURE-

Martin Schwenke (31):
  ctdb tool: fix thinko in nodestatus command output
  Tests - simple - make disconnected node tests handle changed error message
  Tests - functions/environment - set and use $CTDB_NODES
  Eventscript functions - add new function die()
  Eventscripts - redesign and rewrite 13.per_ip_routing
  Eventscript functions - remove functions only used by 13.per_ip_routing
  Eventscript functions - remove now-unused route/IP re-add script logic
  Eventscript functions - no longer require interface_modify.sh
  Eventscript support - Remove unused interface_modify.sh
  Eventscript tests - make error message consistent with recent change
  Eventscript tests - make ip command stub vaguely maintainable
  Eventscript tests - implement ip rule in stub
  Eventscript tests - implement ip route in stub
  Eventscripts - 13.per_ip_routing should use dirname not basename for mkdir
  Eventscripts - use set_proc() rather than accessing /proc directly
  Eventscripts - make 13.per_ip_routing try harder to find public_addresses
  Eventscripts - make 13.per_ip_routing fail gracefully if config is missing
  Tests - eventscripts - ip stub command should print errors to stderr
  Tests - eventscripts - add -A/-D options
  Tests - eventscripts - more public IPs in default setup
  Tests - eventscripts - stub ctdb command updates
  Tests - eventscripts - fix hardcoding error
  Update .gitignore for eventscript testing fu.
  Tests - eventscripts - add support functions for policy routing testing
  Tests - eventscripts - improved support functions for handling public IPs
  Tests - eventscripts - rationalise CTDB fakery support
  Tests - eventscripts - change summary columns
  Tests - eventscripts - new function simple_test_command()
  Tests - eventscripts - initial policy routing tests
  Undo damage done by d8d37493478a26c5f1809a5f3df89ffd6e149281
  Add policy routing documentation to ctdbd(1).

Ronnie Sahlberg (23):
  READONLY: when updating a remote node to revoke a delegation, make sure 
we dont create the record if it doesnt already exist
  ReadOnly: Only restrict the readonly sanity/paranoia check that only 
allows
  Test: add test tool and test that UPDATE_RECORD works on persistent 
databases
  Merge remote branch 'martins/master'
  add max hop count buckets to see how bad hopcounts are
  FETCH COLLAPSE : Change the fetch-lock collapse to collapse ALL fetches, 
including fetch-locks into a single command in flight per record. Also add a 
tunable to enable/disable this optimization for hot records
  DOC document the FetchCollapse tunable
  CTDB: change how the hop count buckets are displayed in ctdb statistics
  STATISTICS: add per-db hop count statistics
  LACOUNT:  Add back lacount mechanism to defer migrating a fetched/read 
copy until after default of 20 consecutive requests from the same node
  DOC: Document the MaxLACount tunable variable
  STICKY: add prototype code to make records stick to a node to calm down 
if they are found to be very hot and accessed by a lot of clients.
  STICKY   document the sticky record parameters
  DOC  regenerate the manpage
  Make NoIPFailback a node local setting. Nodes that have NoIPFailback set 
to !0 can not takeover new ip addresses during failover.
  Make NoIPFailback a node local setting. Nodes that have NoIPFailback set 
to !0 can not takeover new ip addresses during failover.
  NoIPFailback: Exclude nodes which have NoIPFailback as failback targets 
during reallocation
  NoIPFailback: Exclude nodes which have NoIPFailback as failback targets 
during reallocation
  NoIPTakeover: change the tunable name for the dont allow failing 
addresses over onto the node to NoIPTakeover
  NoIPTakeover: change the tunable name for the dont allow failing 
addresses over onto the node to NoIPTakeover
  DOC: regenerate manpages
  DOC: regenerate manpages
  Merge remote branch 'martins/policy_routing' into 1.13

[SCM] CTDB repository - annotated tag ctdb-1.42 created - ctdb-1.42

2012-05-02 Thread Ronnie Sahlberg
The annotated tag, ctdb-1.42 has been created
at  22a0df14312c4bed63dab09f952c98894d701810 (tag)
   tagging  16282b5b2b8fb14454ab61a7ff7c651528a4a117 (commit)
  replaces  ctdb-1.41
 tagged by  Ronnie Sahlberg
on  Thu May 3 11:29:09 2012 +1000

- Log -
Tag for 1.42
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)

iQEcBAABAgAGBQJPod9vAAoJEB6eS+vwPaeG+PMIAPYnvjDF9SnXMT9xwdGbkTd+
2IvJbRpgyTFqfUBSQirw/5ACZK8ZXWy7GrdlBIzkVIU5w5dZH4/QCZqLN2hdcnpQ
5aNfj8sQEOOe5hL4QNXhM2IkiKmuVu0IOLpRHZDnBMIfPpinlN3jHugzX/Q/p/Ny
y555ytQQYRFyfDWxyLjQ7QaJGzOJ6lIcgHMuvz3tnIg88YuogpmJYfNV5x2oOyky
UCajnGzIjvLa5+NtIMijGLGUivXfNQD9KbSDMomvbZnbl9op+jAWldQgN9MlGZI0
iDNmk4MdoAJ9NEkYM534aAgssj2ttiJkHFjm1Tg3lSdY7GRiMh1Kb2G7TnQ5gdk=
=5vAP
-END PGP SIGNATURE-

Amitay Isaacs (19):
  build: Add rules to create ctags/etags
  packaging: Setup directories for rpmbuild
  build: Remove re-definition of same variable
  build: Display correct LIB_FLAGS while building
  build: Use system talloc library if available
  build: Use system tevent library if available
  build: Use system tdb library if available
  recovery: Add prototypes for tdb internal functions
  build: Substitute POPT macros once and reuse variables
  includes: Move special tevent defines from tevent.h to includes.h
  Remove explicit include of lib/tevent/tevent.h.
  ctdb_test: Remove faked wrappers for tevent functions in stub testing
  lib/tevent: Remove local modifications to tevent
  lib/tevent: Remove the files required to build tevent as a library
  lib/tevent: Sync tevent from samba git tree
  lib/talloc: Remove the files required to build talloc as a library
  lib/talloc: Sync talloc from samba git tree
  lib/tdb: Remove the files required to build tdb as a library
  lib/tdb: Sync tdb from samba git tree

Martin Schwenke (3):
  Undo damage done by d8d37493478a26c5f1809a5f3df89ffd6e149281
  Add policy routing documentation to ctdbd(1).
  Initscript - add backup of corrupt non-persistent databases

Ronnie Sahlberg (12):
  Merge remote branch 'martins/policy_routing'
  CTDB: add back the rebalanceip command
  Add test for NoIPTakeover
  Add test for NoIPTakeover
  Merge remote branch 'amitay/tevent-sync' into 1.13
  Merge remote branch 'amitay/tdb-sync' into 1.13
  Merge remote branch 'amitay/talloc-sync' into 1.13
  Merge branch '1.13' of 10.1.1.27:/shared/ctdb/ctdb-master into 1.13
  Add new control to reload the public ip address file on a node
  RELOADIPS: simplify the reloadips code a bit
  DOC: document the reloadips command
  New version 1.42

---


-- 
CTDB repository


[SCM] CTDB repository - branch 1.13 updated - ctdb-1.42-1-gb550f46

2012-05-02 Thread Ronnie Sahlberg
The branch, 1.13 has been updated
   via  b550f4644ea81b7010d401e46cf803b13f16e3c1 (commit)
  from  16282b5b2b8fb14454ab61a7ff7c651528a4a117 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=1.13


- Log -
commit b550f4644ea81b7010d401e46cf803b13f16e3c1
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Thu May 3 11:42:41 2012 +1000

Track all child process so we never send a signal to an unrelated process 
(our child died  and kernel wrapped the pid-space and reused the pid for a 
different process

Wrap all creation of child processes inside ctdb_fork() which is used to 
track all processes we have spawned.
Capture SIGCHLD to track also which child processes have terminated.

Wrap kill() inside ctdb_kill() and make sure that we never send a !0 signal 
to a child process pid that has already terminated (and might have been 
replaced with a

---

Summary of changes:
 Makefile.in |2 +-
 common/ctdb_fork.c  |  124 +++
 common/ctdb_util.c  |   18 --
 include/ctdb_private.h  |5 ++
 server/ctdb_call.c  |4 +-
 server/ctdb_daemon.c|   39 ++---
 server/ctdb_freeze.c|6 +-
 server/ctdb_lockwait.c  |4 +-
 server/ctdb_recover.c   |6 +-
 server/ctdb_recoverd.c  |   12 ++--
 server/ctdb_takeover.c  |6 +-
 server/ctdb_traverse.c  |2 +-
 server/ctdb_update_record.c |6 +-
 server/ctdb_vacuum.c|2 +-
 server/eventscript.c|6 +-
 tests/src/ctdb_test.c   |1 +
 tests/src/ctdbd_test.c  |1 +
 17 files changed, 165 insertions(+), 79 deletions(-)
 create mode 100644 common/ctdb_fork.c


Changeset truncated at 500 lines:

diff --git a/Makefile.in b/Makefile.in
index 1794be5..26eeac8 100755
--- a/Makefile.in
+++ b/Makefile.in
@@ -68,7 +68,7 @@ UTIL_OBJ = lib/util/idtree.o lib/util/db_wrap.o 
lib/util/strlist.o lib/util/util
 CTDB_COMMON_OBJ =  common/ctdb_io.o common/ctdb_util.o \
common/ctdb_ltdb.o common/ctdb_message.o common/cmdline.o  \
lib/util/debug.o common/rb_tree.o @CTDB_SYSTEM_OBJ@ 
common/system_common.o \
-   common/ctdb_logging.c
+   common/ctdb_logging.c common/ctdb_fork.o
 
 CTDB_LIB_OBJ = libctdb/ctdb.o libctdb/io_elem.o libctdb/local_tdb.o \
libctdb/messages.o libctdb/sync.o libctdb/control.o \
diff --git a/common/ctdb_fork.c b/common/ctdb_fork.c
new file mode 100644
index 000..81055c5
--- /dev/null
+++ b/common/ctdb_fork.c
@@ -0,0 +1,124 @@
+/* 
+   functions to track and manage processes
+
+   Copyright (C) Ronnie Sahlberg 2012
+
+   This program is free software; you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published by
+   the Free Software Foundation; either version 3 of the License, or
+   (at your option) any later version.
+   
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+   
+   You should have received a copy of the GNU General Public License
+   along with this program; if not, see http://www.gnu.org/licenses/.
+*/
+
+#include includes.h
+#include system/wait.h
+#include ../include/ctdb_client.h
+#include ../include/ctdb_private.h
+#include ../common/rb_tree.h
+
+/*
+ * This function forks a child process and drops the realtime 
+ * scheduler for the child process.
+ */
+pid_t ctdb_fork(struct ctdb_context *ctdb)
+{
+   pid_t pid;
+   char *process;
+
+   pid = fork();
+   if (pid == -1) {
+   return -1;
+   }
+   if (pid == 0) {
+   if (ctdb-do_setsched) {
+   ctdb_restore_scheduler(ctdb);
+   }
+   ctdb-can_send_controls = false;
+   return 0;
+   }
+
+   if (getpid() != ctdb-ctdbd_pid) {
+   return pid;
+   }
+
+   process = talloc_asprintf(ctdb-child_processes, process:%d, 
(int)pid);
+   trbt_insert32(ctdb-child_processes, pid, process);
+
+   return pid;
+}
+
+
+
+static void ctdb_sigchld_handler(struct tevent_context *ev,
+   struct tevent_signal *te, int signum, int count,
+   void *dont_care, 
+   void *private_data)
+{
+   struct ctdb_context *ctdb = talloc_get_type(private_data, struct 
ctdb_context);
+   int status;
+   pid_t pid = -1;
+
+   while (pid != 0) {
+   pid = waitpid(-1, status, WNOHANG);
+   if (pid == -1) {
+   DEBUG(DEBUG_ERR, (__location__  waitpid() returned 
error. errno:%d\n, errno));
+   return;
+   }
+   if (pid  0) {
+   char *process

[SCM] CTDB repository - branch master updated - ctdb-1.13-154-gf73a4b1

2012-05-02 Thread Ronnie Sahlberg
The branch, master has been updated
   via  f73a4b1495830bcdd094a93732a89dd53b3c2f78 (commit)
  from  90e8c4dc7d89c460ad2da5a9af1cef006db3b1c0 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=master


- Log -
commit f73a4b1495830bcdd094a93732a89dd53b3c2f78
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Thu May 3 11:42:41 2012 +1000

Track all child process so we never send a signal to an unrelated process 
(our child died  and kernel wrapped the pid-space and reused the pid for a 
different process

Wrap all creation of child processes inside ctdb_fork() which is used to 
track all processes we have spawned.
Capture SIGCHLD to track also which child processes have terminated.

Wrap kill() inside ctdb_kill() and make sure that we never send a !0 signal 
to a child process pid that has already terminated (and might have been 
replaced with a

---

Summary of changes:
 Makefile.in |2 +-
 common/ctdb_fork.c  |  124 +++
 common/ctdb_util.c  |   18 --
 include/ctdb_private.h  |5 ++
 server/ctdb_call.c  |4 +-
 server/ctdb_daemon.c|   39 ++---
 server/ctdb_freeze.c|6 +-
 server/ctdb_lockwait.c  |4 +-
 server/ctdb_recover.c   |6 +-
 server/ctdb_recoverd.c  |   12 ++--
 server/ctdb_takeover.c  |6 +-
 server/ctdb_traverse.c  |2 +-
 server/ctdb_update_record.c |6 +-
 server/ctdb_vacuum.c|2 +-
 server/eventscript.c|6 +-
 tests/src/ctdb_test.c   |1 +
 tests/src/ctdbd_test.c  |1 +
 17 files changed, 165 insertions(+), 79 deletions(-)
 create mode 100644 common/ctdb_fork.c


Changeset truncated at 500 lines:

diff --git a/Makefile.in b/Makefile.in
index 843a948..ecf03f2 100755
--- a/Makefile.in
+++ b/Makefile.in
@@ -68,7 +68,7 @@ UTIL_OBJ = lib/util/idtree.o lib/util/db_wrap.o 
lib/util/strlist.o lib/util/util
 CTDB_COMMON_OBJ =  common/ctdb_io.o common/ctdb_util.o \
common/ctdb_ltdb.o common/ctdb_message.o common/cmdline.o  \
lib/util/debug.o common/rb_tree.o @CTDB_SYSTEM_OBJ@ 
common/system_common.o \
-   common/ctdb_logging.c
+   common/ctdb_logging.c common/ctdb_fork.o
 
 CTDB_LIB_OBJ = libctdb/ctdb.o libctdb/io_elem.o libctdb/local_tdb.o \
libctdb/messages.o libctdb/sync.o libctdb/control.o \
diff --git a/common/ctdb_fork.c b/common/ctdb_fork.c
new file mode 100644
index 000..81055c5
--- /dev/null
+++ b/common/ctdb_fork.c
@@ -0,0 +1,124 @@
+/* 
+   functions to track and manage processes
+
+   Copyright (C) Ronnie Sahlberg 2012
+
+   This program is free software; you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published by
+   the Free Software Foundation; either version 3 of the License, or
+   (at your option) any later version.
+   
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+   
+   You should have received a copy of the GNU General Public License
+   along with this program; if not, see http://www.gnu.org/licenses/.
+*/
+
+#include includes.h
+#include system/wait.h
+#include ../include/ctdb_client.h
+#include ../include/ctdb_private.h
+#include ../common/rb_tree.h
+
+/*
+ * This function forks a child process and drops the realtime 
+ * scheduler for the child process.
+ */
+pid_t ctdb_fork(struct ctdb_context *ctdb)
+{
+   pid_t pid;
+   char *process;
+
+   pid = fork();
+   if (pid == -1) {
+   return -1;
+   }
+   if (pid == 0) {
+   if (ctdb-do_setsched) {
+   ctdb_restore_scheduler(ctdb);
+   }
+   ctdb-can_send_controls = false;
+   return 0;
+   }
+
+   if (getpid() != ctdb-ctdbd_pid) {
+   return pid;
+   }
+
+   process = talloc_asprintf(ctdb-child_processes, process:%d, 
(int)pid);
+   trbt_insert32(ctdb-child_processes, pid, process);
+
+   return pid;
+}
+
+
+
+static void ctdb_sigchld_handler(struct tevent_context *ev,
+   struct tevent_signal *te, int signum, int count,
+   void *dont_care, 
+   void *private_data)
+{
+   struct ctdb_context *ctdb = talloc_get_type(private_data, struct 
ctdb_context);
+   int status;
+   pid_t pid = -1;
+
+   while (pid != 0) {
+   pid = waitpid(-1, status, WNOHANG);
+   if (pid == -1) {
+   DEBUG(DEBUG_ERR, (__location__  waitpid() returned 
error. errno:%d\n, errno));
+   return;
+   }
+   if (pid  0) {
+   char

[SCM] CTDB repository - branch 1.13 updated - ctdb-1.13-101-g1412e98

2012-04-30 Thread Ronnie Sahlberg
The branch, 1.13 has been updated
   via  1412e98221dd2080e553b32af0c2906c3c511b86 (commit)
  from  c5c9507ea1e1836d51c2d4291721901bb6875a04 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=1.13


- Log -
commit 1412e98221dd2080e553b32af0c2906c3c511b86
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Mon Apr 30 15:50:44 2012 +1000

Add new control to reload the public ip address file on a node

Also add a method to use the recovery master/daemon to reload the public 
ips on all nodes in the cluster.
Reloading the public ips on all node sin the cluster is only suported if 
all nodes in the cluster are available and healthy.

---

Summary of changes:
 include/ctdb_private.h  |   18 -
 include/ctdb_protocol.h |6 +
 server/ctdb_control.c   |4 +
 server/ctdb_daemon.c|6 +-
 server/ctdb_recoverd.c  |   89 +
 server/ctdb_takeover.c  |  244 +-
 tools/ctdb.c|   91 ++
 7 files changed, 450 insertions(+), 8 deletions(-)


Changeset truncated at 500 lines:

diff --git a/include/ctdb_private.h b/include/ctdb_private.h
index 657bb62..6111910 100644
--- a/include/ctdb_private.h
+++ b/include/ctdb_private.h
@@ -497,6 +497,10 @@ struct ctdb_context {
 
/* list of event script callback functions that are active */
struct event_script_callback *script_callbacks;
+
+   struct ctdb_reloadips_handle *reload_ips;
+
+   const char *public_addresses_file;
 };
 
 struct ctdb_db_context {
@@ -1135,7 +1139,7 @@ int ctdb_sys_send_tcp(const ctdb_sock_addr *dest,
  const ctdb_sock_addr *src,
  uint32_t seq, uint32_t ack, int rst);
 
-int ctdb_set_public_addresses(struct ctdb_context *ctdb, const char *alist);
+int ctdb_set_public_addresses(struct ctdb_context *ctdb);
 int ctdb_set_single_public_ip(struct ctdb_context *ctdb,
  const char *iface,
  const char *ip);
@@ -1470,4 +1474,16 @@ int32_t ctdb_control_get_db_statistics(struct 
ctdb_context *ctdb,
 
 int ctdb_set_db_sticky(struct ctdb_context *ctdb, struct ctdb_db_context 
*ctdb_db);
 
+/*
+  description for a message to reload all ips via recovery master/daemon
+ */
+struct reloadips_all_reply {
+   uint32_t pnn;
+   uint64_t srvid;
+};
+
+int32_t ctdb_control_reload_public_ips(struct ctdb_context *ctdb, struct 
ctdb_req_control *c, bool *async_reply);
+
+int ctdb_start_monitoring_interfaces(struct ctdb_context *ctdb);
+
 #endif
diff --git a/include/ctdb_protocol.h b/include/ctdb_protocol.h
index cb34fd5..5c787ff 100644
--- a/include/ctdb_protocol.h
+++ b/include/ctdb_protocol.h
@@ -135,6 +135,11 @@ struct ctdb_call_info {
 */
 #define CTDB_SRVID_REBALANCE_NODE 0xFB01LL
 
+/*
+   a message handler ID meaning to ask recovery master to reload all ips
+ */
+#define CTDB_SRVID_RELOAD_ALL_IPS 0xFB02LL
+
 /* A message id to ask the recovery daemon to temporarily disable the
public ip checks
 */
@@ -384,6 +389,7 @@ enum ctdb_controls {CTDB_CONTROL_PROCESS_EXISTS  = 
0,
CTDB_CONTROL_TRAVERSE_START_EXT  = 131,
CTDB_CONTROL_GET_DB_STATISTICS   = 132,
CTDB_CONTROL_SET_DB_STICKY   = 133,
+   CTDB_CONTROL_RELOAD_PUBLIC_IPS   = 134,
 };
 
 /*
diff --git a/server/ctdb_control.c b/server/ctdb_control.c
index 0921082..2a16a23 100644
--- a/server/ctdb_control.c
+++ b/server/ctdb_control.c
@@ -644,6 +644,10 @@ static int32_t ctdb_control_dispatch(struct ctdb_context 
*ctdb,
CHECK_CONTROL_DATA_SIZE(sizeof(uint32_t));
return ctdb_control_get_db_statistics(ctdb, *(uint32_t 
*)indata.dptr, outdata);
 
+   case CTDB_CONTROL_RELOAD_PUBLIC_IPS:
+   CHECK_CONTROL_DATA_SIZE(0);
+   return ctdb_control_reload_public_ips(ctdb, c, async_reply);
+
default:
DEBUG(DEBUG_CRIT,(__location__  Unknown CTDB control opcode 
%u\n, opcode));
return -1;
diff --git a/server/ctdb_daemon.c b/server/ctdb_daemon.c
index 5002629..192dbff 100644
--- a/server/ctdb_daemon.c
+++ b/server/ctdb_daemon.c
@@ -1161,11 +1161,15 @@ int ctdb_start_daemon(struct ctdb_context *ctdb, bool 
do_fork, bool use_syslog,
ctdb_fatal(ctdb, transport failed to initialise);
}
if (public_address_list) {
-   ret = ctdb_set_public_addresses(ctdb, public_address_list);
+   ctdb-public_addresses_file = public_address_list;
+   ret = ctdb_set_public_addresses(ctdb);
if (ret == -1) {
DEBUG(DEBUG_ALERT,(Unable to setup public address 
list\n));
exit(1);
}
+   if (ctdb

[SCM] CTDB repository - branch master updated - ctdb-1.13-147-ge9fad1f

2012-04-30 Thread Ronnie Sahlberg
The branch, master has been updated
   via  e9fad1ff82e6fb9e6bf615986c41c746fd415058 (commit)
   via  a0afb4195caab39891a304b8b4eadd94cab7c4a7 (commit)
   via  66d0b913301c3b1037278bcaa0452ecbe07248df (commit)
   via  1c37abac5fd18c139aa55989d0fb6b475f4c22f1 (commit)
   via  047d97dc84a5943556efb31bb2856fd04835fffa (commit)
   via  0c674efd19368d41d9cc28909d2b16c1af54c86c (commit)
   via  df732ca8e27f0f3417b4d5d259157bd0e0632124 (commit)
   via  12c9986059cacda819e669fa77d613b408c62599 (commit)
   via  eddfc02335202d853e29a89f08225ab44cac8a25 (commit)
   via  3219f221a858e499f084b8beb44610537312602b (commit)
   via  b430a5f61cfdc28e967890602ce6dac0e922c07d (commit)
   via  734cbd7def23236d418ebc6d813a748a84900101 (commit)
   via  78f6f209eb15102f4e8baa9106583df0ef4bb693 (commit)
   via  e6bfd3bdb3a35b2e7e7c41a6f37976772a54e3ce (commit)
   via  301491802eec0e49e108f5aae7d7be379703d72c (commit)
   via  8e4ec9a40bb7d392d7474b067a74fa121c069007 (commit)
   via  4aa879466dd46cb4e8710edbbaac1276521e475b (commit)
   via  709cd5dfeb630096a1cd2062da666fee9ddca715 (commit)
   via  fbabc05569d91bf033f474d89f51b49490943332 (commit)
   via  246809af64c03d26288abff5907ed46614e72b15 (commit)
   via  bef03b32f8ec114a64c55b25951aeeb939d87b33 (commit)
   via  95947123eee687ed83eebc567397a5bc9d55a945 (commit)
   via  6eb7fa572e7fc212332ddd68793e3f35161baf7c (commit)
   via  704a3e3b83aff63e8f7b0650c141776ed2c1f047 (commit)
   via  79e979d67e3a2a5a13ef8fd2ef8f56331ec51558 (commit)
   via  a0a7759d47ef5de4a8214273e39c50fb1f6e2e0c (commit)
   via  eef162ceaac6853bf1ff622691901289280bc25f (commit)
   via  d714cf7924674a7a0eb6d585eb74a6a4df26fc12 (commit)
   via  dc71294a33e88baa4e85fa1fa66cab58a83c2607 (commit)
   via  f45295a3005474957852d0e7a5c3807e30ab519d (commit)
   via  6136ab02db261b26a2a58b526c913e37e8146841 (commit)
   via  b5308142d03332d6d4e0c3b77283c772462fbb23 (commit)
   via  66a7fece867966528689d2784a284e32d687a0f5 (commit)
   via  72ecae61c43b318ec94b527a12cbb0a382e8c3db (commit)
   via  9542e770a9780740b49122f1f52f08b32eca4b35 (commit)
   via  bfa1d6638d3e116640eb4e3bb71b21ba6ef8cae5 (commit)
   via  79adb50b3ce3873c3baf9e6715c1d1c3f181ce43 (commit)
  from  8052ee0a6bda3fa88501d77b2d53315be2b75ec1 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=master


- Log -
commit e9fad1ff82e6fb9e6bf615986c41c746fd415058
Author: Martin Schwenke mar...@meltin.net
Date:   Fri Apr 27 17:19:45 2012 +1000

tests: Update README

Signed-off-by: Martin Schwenke mar...@meltin.net

commit a0afb4195caab39891a304b8b4eadd94cab7c4a7
Author: Martin Schwenke mar...@meltin.net
Date:   Fri Apr 27 14:17:52 2012 +1000

tests/eventscripts: $CTDB_ETCDIR should be in $TEST_VAR_DIR

The policy routing tests modify /etc/iproute2/rt_tables, so this
directory should not be in the installation area.

Instead the contents of tests/eventscripts/etc are copied into a place
under $TEST_VAR_DIR where the directory can be modified with gay
abandon.

Signed-off-by: Martin Schwenke mar...@meltin.net

commit 66d0b913301c3b1037278bcaa0452ecbe07248df
Author: Martin Schwenke mar...@meltin.net
Date:   Fri Apr 27 14:16:16 2012 +1000

tests/eventscripts: Use a canned ctdb.sysconfig rather than a link

The link is hard to manage and has no real advantage.

The canned config is 2 of the 3 currently non-comment/whitespace lines
in config/ctdb.sysconfig.

Signed-off-by: Martin Schwenke mar...@meltin.net

commit 1c37abac5fd18c139aa55989d0fb6b475f4c22f1
Author: Martin Schwenke mar...@meltin.net
Date:   Fri Apr 27 14:15:06 2012 +1000

tests/eventscripts: Explicitly set CTDB_SERVICE_AUTOSTARTSTOP=yes

No use depending on defaults in the initscript - the tests should
control what is tested.

Signed-off-by: Martin Schwenke mar...@meltin.net

commit 047d97dc84a5943556efb31bb2856fd04835fffa
Author: Martin Schwenke mar...@meltin.net
Date:   Tue Apr 24 17:17:59 2012 +1000

tests/eventscripts: Be more defensive about removing the var directory

Could add -r but that might avoid error.  The only time it doesn't
exist is the 1st time a top-level var directory is used.

Signed-off-by: Martin Schwenke mar...@meltin.net

commit 0c674efd19368d41d9cc28909d2b16c1af54c86c
Author: Martin Schwenke mar...@meltin.net
Date:   Tue Apr 24 17:17:07 2012 +1000

Eventscripts: fix basename - dirname typo

I fixed one of these previously but didn't notice this one...  :-(

Signed-off-by: Martin Schwenke mar...@meltin.net

commit df732ca8e27f0f3417b4d5d259157bd0e0632124
Author: Martin Schwenke mar...@meltin.net
Date:   Tue Apr 24 16:59:38 2012 +1000

tests/eventscripts: Add sanity check to esnure events.d/ can be found

   

[SCM] CTDB repository - branch master updated - ctdb-1.13-148-g05603e9

2012-04-30 Thread Ronnie Sahlberg
The branch, master has been updated
   via  05603e914f8c12618d7e06943c0f7df207f645b0 (commit)
  from  e9fad1ff82e6fb9e6bf615986c41c746fd415058 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=master


- Log -
commit 05603e914f8c12618d7e06943c0f7df207f645b0
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Mon Apr 30 15:50:44 2012 +1000

Add new control to reload the public ip address file on a node

Also add a method to use the recovery master/daemon to reload the public 
ips on all nodes in the cluster.
Reloading the public ips on all node sin the cluster is only suported if 
all nodes in the cluster are available and healthy.

---

Summary of changes:
 include/ctdb_private.h  |   18 -
 include/ctdb_protocol.h |6 +
 server/ctdb_control.c   |4 +
 server/ctdb_daemon.c|6 +-
 server/ctdb_recoverd.c  |   89 +
 server/ctdb_takeover.c  |  246 +--
 tools/ctdb.c|   91 +
 7 files changed, 450 insertions(+), 10 deletions(-)


Changeset truncated at 500 lines:

diff --git a/include/ctdb_private.h b/include/ctdb_private.h
index 657bb62..6111910 100644
--- a/include/ctdb_private.h
+++ b/include/ctdb_private.h
@@ -497,6 +497,10 @@ struct ctdb_context {
 
/* list of event script callback functions that are active */
struct event_script_callback *script_callbacks;
+
+   struct ctdb_reloadips_handle *reload_ips;
+
+   const char *public_addresses_file;
 };
 
 struct ctdb_db_context {
@@ -1135,7 +1139,7 @@ int ctdb_sys_send_tcp(const ctdb_sock_addr *dest,
  const ctdb_sock_addr *src,
  uint32_t seq, uint32_t ack, int rst);
 
-int ctdb_set_public_addresses(struct ctdb_context *ctdb, const char *alist);
+int ctdb_set_public_addresses(struct ctdb_context *ctdb);
 int ctdb_set_single_public_ip(struct ctdb_context *ctdb,
  const char *iface,
  const char *ip);
@@ -1470,4 +1474,16 @@ int32_t ctdb_control_get_db_statistics(struct 
ctdb_context *ctdb,
 
 int ctdb_set_db_sticky(struct ctdb_context *ctdb, struct ctdb_db_context 
*ctdb_db);
 
+/*
+  description for a message to reload all ips via recovery master/daemon
+ */
+struct reloadips_all_reply {
+   uint32_t pnn;
+   uint64_t srvid;
+};
+
+int32_t ctdb_control_reload_public_ips(struct ctdb_context *ctdb, struct 
ctdb_req_control *c, bool *async_reply);
+
+int ctdb_start_monitoring_interfaces(struct ctdb_context *ctdb);
+
 #endif
diff --git a/include/ctdb_protocol.h b/include/ctdb_protocol.h
index cb34fd5..5c787ff 100644
--- a/include/ctdb_protocol.h
+++ b/include/ctdb_protocol.h
@@ -135,6 +135,11 @@ struct ctdb_call_info {
 */
 #define CTDB_SRVID_REBALANCE_NODE 0xFB01LL
 
+/*
+   a message handler ID meaning to ask recovery master to reload all ips
+ */
+#define CTDB_SRVID_RELOAD_ALL_IPS 0xFB02LL
+
 /* A message id to ask the recovery daemon to temporarily disable the
public ip checks
 */
@@ -384,6 +389,7 @@ enum ctdb_controls {CTDB_CONTROL_PROCESS_EXISTS  = 
0,
CTDB_CONTROL_TRAVERSE_START_EXT  = 131,
CTDB_CONTROL_GET_DB_STATISTICS   = 132,
CTDB_CONTROL_SET_DB_STICKY   = 133,
+   CTDB_CONTROL_RELOAD_PUBLIC_IPS   = 134,
 };
 
 /*
diff --git a/server/ctdb_control.c b/server/ctdb_control.c
index 0921082..2a16a23 100644
--- a/server/ctdb_control.c
+++ b/server/ctdb_control.c
@@ -644,6 +644,10 @@ static int32_t ctdb_control_dispatch(struct ctdb_context 
*ctdb,
CHECK_CONTROL_DATA_SIZE(sizeof(uint32_t));
return ctdb_control_get_db_statistics(ctdb, *(uint32_t 
*)indata.dptr, outdata);
 
+   case CTDB_CONTROL_RELOAD_PUBLIC_IPS:
+   CHECK_CONTROL_DATA_SIZE(0);
+   return ctdb_control_reload_public_ips(ctdb, c, async_reply);
+
default:
DEBUG(DEBUG_CRIT,(__location__  Unknown CTDB control opcode 
%u\n, opcode));
return -1;
diff --git a/server/ctdb_daemon.c b/server/ctdb_daemon.c
index 5002629..192dbff 100644
--- a/server/ctdb_daemon.c
+++ b/server/ctdb_daemon.c
@@ -1161,11 +1161,15 @@ int ctdb_start_daemon(struct ctdb_context *ctdb, bool 
do_fork, bool use_syslog,
ctdb_fatal(ctdb, transport failed to initialise);
}
if (public_address_list) {
-   ret = ctdb_set_public_addresses(ctdb, public_address_list);
+   ctdb-public_addresses_file = public_address_list;
+   ret = ctdb_set_public_addresses(ctdb);
if (ret == -1) {
DEBUG(DEBUG_ALERT,(Unable to setup public address 
list\n));
exit(1);
}
+   if (ctdb

[SCM] CTDB repository - branch master updated - ctdb-1.13-151-g2a3d224

2012-04-30 Thread Ronnie Sahlberg
The branch, master has been updated
   via  2a3d22431948f849c93762a6ab49344acb687feb (commit)
   via  6a92fc2b8da2bba98dca29b781ab459ba4e879a5 (commit)
   via  c642b452e7b0272e32c02509ac579b790d9a03e7 (commit)
  from  05603e914f8c12618d7e06943c0f7df207f645b0 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=master


- Log -
commit 2a3d22431948f849c93762a6ab49344acb687feb
Author: Amitay Isaacs ami...@gmail.com
Date:   Tue May 1 12:09:48 2012 +1000

Revert server: locking: Provide a common API for non-blocking locking of 
TDBs

This reverts commit 6a92fc2b8da2bba98dca29b781ab459ba4e879a5.

Reverting incomplete changes to ctdb_lock.c

commit 6a92fc2b8da2bba98dca29b781ab459ba4e879a5
Author: Amitay Isaacs ami...@gmail.com
Date:   Thu Apr 26 15:48:12 2012 +1000

server: locking: Provide a common API for non-blocking locking of TDBs

commit c642b452e7b0272e32c02509ac579b790d9a03e7
Author: Amitay Isaacs ami...@gmail.com
Date:   Mon Apr 23 15:53:29 2012 +1000

tests/simple: Fix typo in the test message

Signed-off-by: Amitay Isaacs ami...@gmail.com

---

Summary of changes:
 tests/simple/75_readonly_records_basic.sh |2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)


Changeset truncated at 500 lines:

diff --git a/tests/simple/75_readonly_records_basic.sh 
b/tests/simple/75_readonly_records_basic.sh
index db5b792..f243ea1 100755
--- a/tests/simple/75_readonly_records_basic.sh
+++ b/tests/simple/75_readonly_records_basic.sh
@@ -71,7 +71,7 @@ try_command_on_node -q 0,1,2 $CTDB_TEST_WRAPPER 
ctdb_fetch_readonly_once /dev/
 # no delegations should have been created
 numreadonly=`try_command_on_node -v all $CTDB_TEST_WRAPPER ctdb cattdb 
test.tdb | grep READONLY | wc -l`
 [ $numreadonly != 0 ]  {
-echo BAD: readonly delegations were created byt the feature is not 
activated on the database
+echo BAD: readonly delegations were created, but the feature is not 
activated on the database
 exit 1
 }
 


-- 
CTDB repository


[SCM] CTDB repository - branch 1.13 updated - ctdb-1.13-102-gd697e75

2012-04-30 Thread Ronnie Sahlberg
The branch, 1.13 has been updated
   via  d697e75605e94bfeb57854995b9af2a558849d56 (commit)
  from  1412e98221dd2080e553b32af0c2906c3c511b86 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=1.13


- Log -
commit d697e75605e94bfeb57854995b9af2a558849d56
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Tue May 1 15:27:12 2012 +1000

RELOADIPS: simplify the reloadips code a bit
and also update the read public address file to not check if the address 
exists already locally when we read if from the child process, to stop it
from spamming the logs with We already host ...
messages

---

Summary of changes:
 include/ctdb_private.h |2 +-
 server/ctdb_daemon.c   |2 +-
 server/ctdb_takeover.c |   71 +---
 3 files changed, 33 insertions(+), 42 deletions(-)


Changeset truncated at 500 lines:

diff --git a/include/ctdb_private.h b/include/ctdb_private.h
index 6111910..d973fcc 100644
--- a/include/ctdb_private.h
+++ b/include/ctdb_private.h
@@ -1139,7 +1139,7 @@ int ctdb_sys_send_tcp(const ctdb_sock_addr *dest,
  const ctdb_sock_addr *src,
  uint32_t seq, uint32_t ack, int rst);
 
-int ctdb_set_public_addresses(struct ctdb_context *ctdb);
+int ctdb_set_public_addresses(struct ctdb_context *ctdb, bool check_addresses);
 int ctdb_set_single_public_ip(struct ctdb_context *ctdb,
  const char *iface,
  const char *ip);
diff --git a/server/ctdb_daemon.c b/server/ctdb_daemon.c
index 192dbff..f946ac6 100644
--- a/server/ctdb_daemon.c
+++ b/server/ctdb_daemon.c
@@ -1162,7 +1162,7 @@ int ctdb_start_daemon(struct ctdb_context *ctdb, bool 
do_fork, bool use_syslog,
}
if (public_address_list) {
ctdb-public_addresses_file = public_address_list;
-   ret = ctdb_set_public_addresses(ctdb);
+   ret = ctdb_set_public_addresses(ctdb, true);
if (ret == -1) {
DEBUG(DEBUG_ALERT,(Unable to setup public address 
list\n));
exit(1);
diff --git a/server/ctdb_takeover.c b/server/ctdb_takeover.c
index 0b86af0..3668305 100644
--- a/server/ctdb_takeover.c
+++ b/server/ctdb_takeover.c
@@ -870,7 +870,8 @@ int32_t ctdb_control_release_ipv4(struct ctdb_context *ctdb,
 
 static int ctdb_add_public_address(struct ctdb_context *ctdb,
   ctdb_sock_addr *addr,
-  unsigned mask, const char *ifaces)
+  unsigned mask, const char *ifaces,
+  bool check_address)
 {
struct ctdb_vnn  *vnn;
uint32_t num = 0;
@@ -916,9 +917,11 @@ static int ctdb_add_public_address(struct ctdb_context 
*ctdb,
vnn-public_address  = *addr;
vnn-public_netmask_bits = mask;
vnn-pnn = -1;
-   if (ctdb_sys_have_ip(addr)) {
-   DEBUG(DEBUG_ERR,(We are already hosting public address '%s'. 
setting PNN to ourself:%d\n, ctdb_addr_to_str(addr), ctdb-pnn));
-   vnn-pnn = ctdb-pnn;
+   if (check_address) {
+   if (ctdb_sys_have_ip(addr)) {
+   DEBUG(DEBUG_ERR,(We are already hosting public address 
'%s'. setting PNN to ourself:%d\n, ctdb_addr_to_str(addr), ctdb-pnn));
+   vnn-pnn = ctdb-pnn;
+   }
}
 
for (i=0; vnn-ifaces[i]; i++) {
@@ -998,7 +1001,7 @@ int ctdb_start_monitoring_interfaces(struct ctdb_context 
*ctdb)
 /*
   setup the public address lists from a file
 */
-int ctdb_set_public_addresses(struct ctdb_context *ctdb)
+int ctdb_set_public_addresses(struct ctdb_context *ctdb, bool check_addresses)
 {
char **lines;
int nlines;
@@ -1050,7 +1053,7 @@ int ctdb_set_public_addresses(struct ctdb_context *ctdb)
talloc_free(lines);
return -1;
}
-   if (ctdb_add_public_address(ctdb, addr, mask, ifaces)) {
+   if (ctdb_add_public_address(ctdb, addr, mask, ifaces, 
check_addresses)) {
DEBUG(DEBUG_CRIT,(Failed to add line %u to the public 
address list\n, i+1));
talloc_free(lines);
return -1;
@@ -3495,7 +3498,7 @@ int32_t ctdb_control_add_public_address(struct 
ctdb_context *ctdb, TDB_DATA inda
return -1;
}
 
-   ret = ctdb_add_public_address(ctdb, pub-addr, pub-mask, 
pub-iface[0]);
+   ret = ctdb_add_public_address(ctdb, pub-addr, pub-mask, 
pub-iface[0], true);
 
if (ret != 0) {
DEBUG(DEBUG_ERR,(__location__  Failed to add public 
address\n));
@@ -3635,34 +3638,25 @@ int update_ip_assignment_tree(struct ctdb_context 
*ctdb, struct

[SCM] CTDB repository - branch master updated - ctdb-1.13-152-g334ea83

2012-04-30 Thread Ronnie Sahlberg
The branch, master has been updated
   via  334ea830f1bf33419f4a1e78f23afd41a852d0f4 (commit)
  from  2a3d22431948f849c93762a6ab49344acb687feb (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=master


- Log -
commit 334ea830f1bf33419f4a1e78f23afd41a852d0f4
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Tue May 1 15:27:12 2012 +1000

RELOADIPS: simplify the reloadips code a bit
and also update the read public address file to not check if the address 
exists already locally when we read if from the child process, to stop it
from spamming the logs with We already host ...
messages

---

Summary of changes:
 include/ctdb_private.h |2 +-
 server/ctdb_daemon.c   |2 +-
 server/ctdb_takeover.c |   71 +---
 3 files changed, 33 insertions(+), 42 deletions(-)


Changeset truncated at 500 lines:

diff --git a/include/ctdb_private.h b/include/ctdb_private.h
index 6111910..d973fcc 100644
--- a/include/ctdb_private.h
+++ b/include/ctdb_private.h
@@ -1139,7 +1139,7 @@ int ctdb_sys_send_tcp(const ctdb_sock_addr *dest,
  const ctdb_sock_addr *src,
  uint32_t seq, uint32_t ack, int rst);
 
-int ctdb_set_public_addresses(struct ctdb_context *ctdb);
+int ctdb_set_public_addresses(struct ctdb_context *ctdb, bool check_addresses);
 int ctdb_set_single_public_ip(struct ctdb_context *ctdb,
  const char *iface,
  const char *ip);
diff --git a/server/ctdb_daemon.c b/server/ctdb_daemon.c
index 192dbff..f946ac6 100644
--- a/server/ctdb_daemon.c
+++ b/server/ctdb_daemon.c
@@ -1162,7 +1162,7 @@ int ctdb_start_daemon(struct ctdb_context *ctdb, bool 
do_fork, bool use_syslog,
}
if (public_address_list) {
ctdb-public_addresses_file = public_address_list;
-   ret = ctdb_set_public_addresses(ctdb);
+   ret = ctdb_set_public_addresses(ctdb, true);
if (ret == -1) {
DEBUG(DEBUG_ALERT,(Unable to setup public address 
list\n));
exit(1);
diff --git a/server/ctdb_takeover.c b/server/ctdb_takeover.c
index 1670f01..f147446 100644
--- a/server/ctdb_takeover.c
+++ b/server/ctdb_takeover.c
@@ -889,7 +889,8 @@ int32_t ctdb_control_release_ipv4(struct ctdb_context *ctdb,
 
 static int ctdb_add_public_address(struct ctdb_context *ctdb,
   ctdb_sock_addr *addr,
-  unsigned mask, const char *ifaces)
+  unsigned mask, const char *ifaces,
+  bool check_address)
 {
struct ctdb_vnn  *vnn;
uint32_t num = 0;
@@ -935,9 +936,11 @@ static int ctdb_add_public_address(struct ctdb_context 
*ctdb,
vnn-public_address  = *addr;
vnn-public_netmask_bits = mask;
vnn-pnn = -1;
-   if (ctdb_sys_have_ip(addr)) {
-   DEBUG(DEBUG_ERR,(We are already hosting public address '%s'. 
setting PNN to ourself:%d\n, ctdb_addr_to_str(addr), ctdb-pnn));
-   vnn-pnn = ctdb-pnn;
+   if (check_address) {
+   if (ctdb_sys_have_ip(addr)) {
+   DEBUG(DEBUG_ERR,(We are already hosting public address 
'%s'. setting PNN to ourself:%d\n, ctdb_addr_to_str(addr), ctdb-pnn));
+   vnn-pnn = ctdb-pnn;
+   }
}
 
for (i=0; vnn-ifaces[i]; i++) {
@@ -1017,7 +1020,7 @@ int ctdb_start_monitoring_interfaces(struct ctdb_context 
*ctdb)
 /*
   setup the public address lists from a file
 */
-int ctdb_set_public_addresses(struct ctdb_context *ctdb)
+int ctdb_set_public_addresses(struct ctdb_context *ctdb, bool check_addresses)
 {
char **lines;
int nlines;
@@ -1069,7 +1072,7 @@ int ctdb_set_public_addresses(struct ctdb_context *ctdb)
talloc_free(lines);
return -1;
}
-   if (ctdb_add_public_address(ctdb, addr, mask, ifaces)) {
+   if (ctdb_add_public_address(ctdb, addr, mask, ifaces, 
check_addresses)) {
DEBUG(DEBUG_CRIT,(Failed to add line %u to the public 
address list\n, i+1));
talloc_free(lines);
return -1;
@@ -3514,7 +3517,7 @@ int32_t ctdb_control_add_public_address(struct 
ctdb_context *ctdb, TDB_DATA inda
return -1;
}
 
-   ret = ctdb_add_public_address(ctdb, pub-addr, pub-mask, 
pub-iface[0]);
+   ret = ctdb_add_public_address(ctdb, pub-addr, pub-mask, 
pub-iface[0], true);
 
if (ret != 0) {
DEBUG(DEBUG_ERR,(__location__  Failed to add public 
address\n));
@@ -3654,34 +3657,25 @@ int update_ip_assignment_tree(struct ctdb_context 
*ctdb, struct

[SCM] CTDB repository - branch master updated - ctdb-1.13-110-g8052ee0

2012-04-25 Thread Ronnie Sahlberg
The branch, master has been updated
   via  8052ee0a6bda3fa88501d77b2d53315be2b75ec1 (commit)
   via  17ff3f240b0d72c72ed28d70fb9aeb3b20c80670 (commit)
   via  fdd5b9ac0a79f6bc7c8a05da2e2f11e002392bcd (commit)
   via  0f40ea2386892ae10b30beeded0e00edf4c019c3 (commit)
   via  a7aace3d165e6cb7d63950107e584edaee12a357 (commit)
   via  670ebada4bafe79bd535c9bb5cc2d5d49d646af3 (commit)
   via  2cd6a7ce648c1d2cc4d5580c3809858c100fa930 (commit)
   via  483459c79884891b3639a8bb865d5e8318cde98c (commit)
   via  efbb7e0b9dcde547b46efd8e3b26c571e4fcf893 (commit)
   via  d1bc4a97ea649f5a07802178591688e438313ff8 (commit)
   via  74539cc619b17ea187e34b5866869212896b0c1a (commit)
   via  0681014ca5ed2a9b56f63fdace7f894beccf8a9a (commit)
   via  9ae4389c2c959c5dcd8395fdae2b25ed7e1e873a (commit)
  from  abe7ebe6c531f912efe4ebc1daade4d2af67574c (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=master


- Log -
commit 8052ee0a6bda3fa88501d77b2d53315be2b75ec1
Merge: 17ff3f240b0d72c72ed28d70fb9aeb3b20c80670 
0f40ea2386892ae10b30beeded0e00edf4c019c3
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Thu Apr 26 08:09:34 2012 +1000

Merge remote branch 'amitay/tdb-sync'

commit 17ff3f240b0d72c72ed28d70fb9aeb3b20c80670
Merge: fdd5b9ac0a79f6bc7c8a05da2e2f11e002392bcd 
483459c79884891b3639a8bb865d5e8318cde98c
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Thu Apr 26 08:09:23 2012 +1000

Merge remote branch 'amitay/tevent-sync'

commit fdd5b9ac0a79f6bc7c8a05da2e2f11e002392bcd
Merge: abe7ebe6c531f912efe4ebc1daade4d2af67574c 
670ebada4bafe79bd535c9bb5cc2d5d49d646af3
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Thu Apr 26 08:09:05 2012 +1000

Merge remote branch 'amitay/talloc-sync'

---

Summary of changes:
 client/ctdb_client.c   |1 -
 common/cmdline.c   |1 -
 common/ctdb_io.c   |1 -
 common/ctdb_logging.c  |1 -
 common/ctdb_ltdb.c |1 -
 common/ctdb_message.c  |1 -
 common/ctdb_util.c |1 -
 common/system_freebsd.c|1 -
 common/system_gnu.c|1 -
 common/system_kfreebsd.c   |1 -
 common/system_linux.c  |1 -
 ib/ibw_ctdb.c  |1 -
 ib/ibw_ctdb_init.c |1 -
 ib/ibwrapper_test.c|1 -
 include/includes.h |7 +
 lib/talloc/ABI/talloc-2.0.7.sigs   |   62 +
 lib/talloc/Makefile.in |   72 -
 lib/talloc/aclocal.m4  |1 -
 lib/talloc/autogen.sh  |   14 -
 lib/talloc/config.guess| 1533 --
 lib/talloc/config.h.in |  702 
 lib/talloc/config.mk   |   13 -
 lib/talloc/config.sub  | 1693 
 lib/talloc/configure.ac|   49 -
 lib/talloc/talloc.3|  500 --
 lib/talloc/talloc.3.html   |  433 -
 lib/talloc/talloc.3.xml|   19 +-
 lib/talloc/talloc.c|  545 ++--
 lib/talloc/talloc.h|   74 +-
 lib/talloc/talloc.pc   |   12 -
 lib/talloc/talloc_guide.txt|   65 +-
 lib/talloc/talloc_testsuite.h  |7 +
 lib/talloc/testsuite.c |  220 +++-
 lib/talloc/testsuite_main.c|3 +-
 lib/tdb/ABI/{tdb-1.2.6.sigs = tdb-1.2.10.sigs}|5 +
 lib/tdb/Makefile.in|   74 -
 lib/tdb/aclocal.m4 |1 -
 lib/tdb/autogen.sh |   14 -
 lib/tdb/build_macros.m4|   14 -
 lib/tdb/common/check.c |   12 +-
 lib/tdb/common/dump.c  |4 +-
 lib/tdb/common/error.c |4 +-
 lib/tdb/common/freelist.c  |4 +-
 lib/tdb/common/freelistcheck.c |2 +-
 lib/tdb/common/hash.c  |   39 +-
 lib/tdb/common/io.c|  152 ++-
 lib/tdb/common/lock.c  |   56 +-
 lib/tdb/common/open.c  |   61 +-
 lib/tdb/common/summary.c

[SCM] CTDB repository - branch 1.13 updated - ctdb-1.13-100-gc5c9507

2012-04-25 Thread Ronnie Sahlberg
The branch, 1.13 has been updated
   via  c5c9507ea1e1836d51c2d4291721901bb6875a04 (commit)
   via  9656584f6b21dbcb816e0a45c5fdba1ddd518777 (commit)
   via  7dbfa76ce65ecad027de8bebee582146cbe0b37c (commit)
   via  457e246bb379c4b07ca0729ebefbb0e067bddd4c (commit)
   via  0f40ea2386892ae10b30beeded0e00edf4c019c3 (commit)
   via  a7aace3d165e6cb7d63950107e584edaee12a357 (commit)
   via  670ebada4bafe79bd535c9bb5cc2d5d49d646af3 (commit)
   via  2cd6a7ce648c1d2cc4d5580c3809858c100fa930 (commit)
   via  483459c79884891b3639a8bb865d5e8318cde98c (commit)
   via  efbb7e0b9dcde547b46efd8e3b26c571e4fcf893 (commit)
   via  d1bc4a97ea649f5a07802178591688e438313ff8 (commit)
   via  74539cc619b17ea187e34b5866869212896b0c1a (commit)
   via  0681014ca5ed2a9b56f63fdace7f894beccf8a9a (commit)
   via  9ae4389c2c959c5dcd8395fdae2b25ed7e1e873a (commit)
   via  7218f17fa93716bf7b612b04284f654a5e95fc53 (commit)
   via  79172330d10ae0d5a7e5bf724959c9e5784716b4 (commit)
   via  83aea6488002aa75dd25752fa0d164afb8b4f300 (commit)
   via  a2ba2c4ec153c8844cf7f5b0de03b2269ec892f3 (commit)
   via  98ac99c4a79fe2ee024890bb27c3ca68dc02d434 (commit)
   via  10ab88a5c822ac403a70a9cd1a6c81020eb91d08 (commit)
   via  2ade83de4e1f318c91489ec9a7f90d1b26f77e1b (commit)
   via  94f1e02406c353c3a0235685acdd107ee5a7f285 (commit)
   via  0cbe7a4f92795203f022252a50a240b8bd4238c3 (commit)
   via  36e690faab3d76919780c8550fe6d49fb6bbfba5 (commit)
   via  45773dab630ee44b39df0dd4b50f43fb1e7bb0ba (commit)
   via  a9e9444a7a78f5ef0e929a8dbd8ca0c68b40acd4 (commit)
   via  00cd75595685dae829758abf1a4cb644af7ed50e (commit)
   via  5cdeb0dd8827b13b06265c75dfac359bcd6ad10c (commit)
   via  7293762046e25a9c96e6417e34e18c214ebc6e55 (commit)
   via  5083b266dd68b292c4275505f3d1b878dbf12f11 (commit)
  from  0489d75a27b63f3f2e69f18400498e667995738a (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=1.13


- Log -
commit c5c9507ea1e1836d51c2d4291721901bb6875a04
Merge: 9656584f6b21dbcb816e0a45c5fdba1ddd518777 
7218f17fa93716bf7b612b04284f654a5e95fc53
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Thu Apr 26 08:40:49 2012 +1000

Merge branch '1.13' of 10.1.1.27:/shared/ctdb/ctdb-master into 1.13

commit 9656584f6b21dbcb816e0a45c5fdba1ddd518777
Merge: 7dbfa76ce65ecad027de8bebee582146cbe0b37c 
670ebada4bafe79bd535c9bb5cc2d5d49d646af3
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Thu Apr 26 08:40:01 2012 +1000

Merge remote branch 'amitay/talloc-sync' into 1.13

commit 7dbfa76ce65ecad027de8bebee582146cbe0b37c
Merge: 457e246bb379c4b07ca0729ebefbb0e067bddd4c 
0f40ea2386892ae10b30beeded0e00edf4c019c3
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Thu Apr 26 08:39:57 2012 +1000

Merge remote branch 'amitay/tdb-sync' into 1.13

commit 457e246bb379c4b07ca0729ebefbb0e067bddd4c
Merge: 0489d75a27b63f3f2e69f18400498e667995738a 
483459c79884891b3639a8bb865d5e8318cde98c
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Thu Apr 26 08:39:52 2012 +1000

Merge remote branch 'amitay/tevent-sync' into 1.13

commit 7218f17fa93716bf7b612b04284f654a5e95fc53
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Thu Apr 12 10:31:17 2012 +1000

Add test for NoIPTakeover

---

Summary of changes:
 Makefile.in|   44 +-
 client/ctdb_client.c   |1 -
 common/cmdline.c   |1 -
 common/ctdb_io.c   |1 -
 common/ctdb_logging.c  |1 -
 common/ctdb_ltdb.c |1 -
 common/ctdb_message.c  |1 -
 common/ctdb_util.c |1 -
 common/system_freebsd.c|1 -
 common/system_gnu.c|1 -
 common/system_kfreebsd.c   |1 -
 common/system_linux.c  |1 -
 ib/ibw_ctdb.c  |1 -
 ib/ibw_ctdb_init.c |1 -
 ib/ibwrapper_test.c|1 -
 include/includes.h |7 +
 lib/talloc/ABI/talloc-2.0.7.sigs   |   62 +
 lib/talloc/Makefile.in |   72 -
 lib/talloc/aclocal.m4  |1 -
 lib/talloc/autogen.sh  |   14 -
 lib/talloc/config.guess| 1533 --
 lib/talloc/config.h.in |  702 
 lib/talloc/config.mk   |   13 -
 lib/talloc/config.sub  | 1693

[SCM] CTDB repository - branch master updated - ctdb-1.13-97-gabe7ebe

2012-04-22 Thread Ronnie Sahlberg
The branch, master has been updated
   via  abe7ebe6c531f912efe4ebc1daade4d2af67574c (commit)
   via  7ff485687891732074c9fc9998502ca197663d02 (commit)
   via  8544162b7a4efa23c92bc0d2774df95bb7f0f463 (commit)
   via  5d6e71cf6e1e0665ec96e7e4e2cfd31f40ebabae (commit)
   via  fd08fc6c88cb80190ce87325867de0391cf1af51 (commit)
   via  eb13507713ba6732271b7c3024bfddbda6da5ffc (commit)
   via  dbec696930327ff07b39282e3084eef4ded064c0 (commit)
   via  f94897ef9504c144937409688fa7a646e0b0ceae (commit)
   via  4c87888b2a559c0d1f285919a42de450a537a02f (commit)
   via  89571585d87b391ff79647cd1f0f6ac193079e72 (commit)
   via  ca9e75eaea9c9b02bb44ca338e28fdae0c8d582c (commit)
   via  a65ba7211ec7f102b32060b173606a61fb5c20a9 (commit)
   via  21df43c74bfcff420fdaf9df5440c25529c543d2 (commit)
   via  89719384a74161ffa0c03602ecdd9e758d521d75 (commit)
   via  43badc5418b9f533398cd579607d9f1fc0f8f417 (commit)
   via  00713eb46cce638339845799bba2da041b3d02fb (commit)
   via  afdaa5f032938d56ff315d9553cb285ebc413c4c (commit)
   via  a7ad94fe9f2e773567dbb6500469dd2dd2f2f04b (commit)
   via  107b465172205cb304549fcffaf36b9416696c15 (commit)
   via  fa54a1095a7504863ed30bff8e0828a8be6dde55 (commit)
   via  89e322562f25dcb8d84357a19e71b2272b2f29a8 (commit)
   via  2aa9bbf3a52dde0707eb06acd91e57c8da5c717f (commit)
   via  67b909a0718d6cfce82ffce0830da3a6ff1f6c4b (commit)
   via  7f8096f56d8274151705ac822b582d972078f8fe (commit)
   via  d5b2ad651495f32091bd33d30871638de0de633a (commit)
   via  b86b947797c51e3576c6b34f547434c3f0aa36f3 (commit)
   via  b81c2c8a2360001fe73f9fd9aa1ba32665fdb0bb (commit)
   via  a108fc14d90b6d552736d19b0ea104f2508dc331 (commit)
   via  15f63ebab9686734f41a6adf38d4a7faa919ac66 (commit)
   via  706e6e3ef6765fbba29e531e590c48fe742b9914 (commit)
   via  98c633ab4824e0e4d28ab2eb2532e16eb859bd94 (commit)
   via  420a10be89c62b4d605f3b972d78fd69dd9f5bb1 (commit)
   via  bdd8b8ab20169936fa723fa77cead1b76b6cd0b5 (commit)
   via  902d8a4709e7a269d2a75522a998f9a60ae13d78 (commit)
   via  261258c1a4268230b2559bc5d8664b1197b22450 (commit)
  from  79172330d10ae0d5a7e5bf724959c9e5784716b4 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=master


- Log -
commit abe7ebe6c531f912efe4ebc1daade4d2af67574c
Merge: 67b909a0718d6cfce82ffce0830da3a6ff1f6c4b 
7ff485687891732074c9fc9998502ca197663d02
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Wed Apr 18 13:48:43 2012 +1000

Merge remote branch 'martins/master'

commit 7ff485687891732074c9fc9998502ca197663d02
Author: Martin Schwenke mar...@meltin.net
Date:   Wed Apr 18 10:37:45 2012 +1000

tests: More unit test factoring/rationalisation and bug fixes

Move some options from eventscripts/run_tests.sh to scripts/run_tests.
Remove the former.

Move some functions from eventscripts/scripts/local.sh to
scripts/unit.sh.

Both of these are modified during move so they are no longer
eventscript-specific.

Tweak */local.sh so that the new functions in unit.sh are used.

Signed-off-by: Martin Schwenke mar...@meltin.net

commit 8544162b7a4efa23c92bc0d2774df95bb7f0f463
Author: Martin Schwenke mar...@meltin.net
Date:   Wed Apr 18 10:35:25 2012 +1000

tests/eventscripts: Share directories must be absolute in eventscript tests

This fixes eventscripts/scripts/local.sh:setup_generic() so that
directories listed in $FAKE_SHARES are absolute instead of potentially
relative (and, therefore, ignored by the NFS share check).

Signed-off-by: Martin Schwenke mar...@meltin.net

commit 5d6e71cf6e1e0665ec96e7e4e2cfd31f40ebabae
Author: Martin Schwenke mar...@meltin.net
Date:   Tue Apr 17 12:57:46 2012 +1000

tests/eventscripts: Tweak an error message in a policy routing test

It currently has my home directory pasted into it.  That can't be
right...  ;-)

Signed-off-by: Martin Schwenke mar...@meltin.net

commit fd08fc6c88cb80190ce87325867de0391cf1af51
Author: Martin Schwenke mar...@meltin.net
Date:   Mon Apr 16 16:27:20 2012 +1000

tests - run_tests needs to expand directories like . and ...

Signed-off-by: Martin Schwenke mar...@meltin.net

commit eb13507713ba6732271b7c3024bfddbda6da5ffc
Author: Martin Schwenke mar...@meltin.net
Date:   Mon Apr 16 16:24:12 2012 +1000

tests/eventscripts - Restructure according to new convention

Signed-off-by: Martin Schwenke mar...@meltin.net

commit dbec696930327ff07b39282e3084eef4ded064c0
Author: Martin Schwenke mar...@meltin.net
Date:   Mon Apr 16 15:58:44 2012 +1000

tests - run_tests ignores trailing '/' on directories

Signed-off-by: Martin Schwenke mar...@meltin.net

commit f94897ef9504c144937409688fa7a646e0b0ceae
Author: Martin Schwenke mar...@meltin.net
Date:   Mon Apr 16 15:53:24 2012

[SCM] CTDB repository - branch master updated - ctdb-1.13-62-g7917233

2012-04-11 Thread Ronnie Sahlberg
The branch, master has been updated
   via  79172330d10ae0d5a7e5bf724959c9e5784716b4 (commit)
  from  83aea6488002aa75dd25752fa0d164afb8b4f300 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=master


- Log -
commit 79172330d10ae0d5a7e5bf724959c9e5784716b4
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Thu Apr 12 10:31:17 2012 +1000

Add test for NoIPTakeover

---

Summary of changes:
 tests/simple/73_tunable_NoIPTakeover.sh |   83 +++
 1 files changed, 83 insertions(+), 0 deletions(-)
 create mode 100755 tests/simple/73_tunable_NoIPTakeover.sh


Changeset truncated at 500 lines:

diff --git a/tests/simple/73_tunable_NoIPTakeover.sh 
b/tests/simple/73_tunable_NoIPTakeover.sh
new file mode 100755
index 000..2f44ce4
--- /dev/null
+++ b/tests/simple/73_tunable_NoIPTakeover.sh
@@ -0,0 +1,83 @@
+#!/bin/bash
+
+test_info()
+{
+cat EOF
+Verify that  'ctdb setvar NoIPTakeover 1' stops ip addresses from being failed 
+over onto the node.
+
+Prerequisites:
+
+* An active CTDB cluster with at least 2 active nodes.
+
+Steps:
+
+1. Verify that the status on all of the ctdb nodes is 'OK'.
+2. Use 'ctdb ip' on one of the nodes to list the IP addresses being
+   served.
+3. Use 'ctdb moveip' to move an address from one node to another.
+4. Verify that the IP is no longer being hosted by the first node and is now 
being hosted by the second node.
+
+Expected results:
+
+* 'ctdb moveip' allows an IP address to be moved between cluster nodes.
+EOF
+}
+
+. ctdb_test_functions.bash
+
+ctdb_test_init $@
+
+cluster_is_healthy
+
+# Reset configuration
+ctdb_restart_when_done
+
+try_command_on_node 0 $CTDB listnodes | wc -l
+num_nodes=$out
+echo There are $num_nodes nodes...
+
+if [ $num_nodes -lt 2 ] ; then
+echo Less than 2 nodes!
+exit 1
+fi
+
+
+echo Wait until the ips are reallocated
+sleep 30
+try_command_on_node -q 0 $CTDB ipreallocate
+
+num=`try_command_on_node -v 1 $CTDB ip | grep -v Public | egrep  1$ | wc 
-l`
+echo Number of addresses on node 1 : $num
+
+
+echo Turning on NoIPTakeover on node 1
+try_command_on_node -q 1 $CTDB setvar NoIPTakeover 1
+try_command_on_node -q 1 $CTDB ipreallocate
+
+echo Disable node 1
+try_command_on_node -q 1 $CTDB disable
+try_command_on_node -q 1 $CTDB ipreallocate
+num=`try_command_on_node -v 1 $CTDB ip | grep -v Public | egrep  1$ | wc 
-l`
+echo Number of addresses on node 1 : $num
+[ $num != 0 ]  {
+echo BAD: node 1 still hosts ip addresses
+exit 1
+}
+
+
+echo Enable node 1 again
+try_command_on_node -q 1 $CTDB enable
+sleep 30
+try_command_on_node -q 1 $CTDB ipreallocate
+try_command_on_node -q 1 $CTDB ipreallocate
+num=`try_command_on_node -v 1 $CTDB ip | grep -v Public | egrep  1$ | wc 
-l`
+echo Number of addresses on node 1 : $num
+[ $num != 0 ]  {
+echo BAD: node took over ip addresses
+exit 1
+}
+
+
+echo OK. ip addresses were not taken over
+exit 0


-- 
CTDB repository


[SCM] CTDB repository - branch 1.13 updated - ctdb-1.13-70-g0489d75

2012-04-02 Thread Ronnie Sahlberg
The branch, 1.13 has been updated
   via  0489d75a27b63f3f2e69f18400498e667995738a (commit)
   via  2899dd5d400bbfcaddc2b8a0e9a405dca979f6d0 (commit)
  from  aaec46211b72f7d6886832301eee169d4b6a5f80 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=1.13


- Log -
commit 0489d75a27b63f3f2e69f18400498e667995738a
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Tue Apr 3 13:45:22 2012 +1000

New version 1.41

commit 2899dd5d400bbfcaddc2b8a0e9a405dca979f6d0
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Tue Apr 3 13:43:21 2012 +1000

CTDB: add back the rebalanceip command

---

Summary of changes:
 packaging/RPM/ctdb.spec.in |2 +
 tools/ctdb.c   |   75 
 2 files changed, 77 insertions(+), 0 deletions(-)


Changeset truncated at 500 lines:

diff --git a/packaging/RPM/ctdb.spec.in b/packaging/RPM/ctdb.spec.in
index f9a5cef..8db59d2 100644
--- a/packaging/RPM/ctdb.spec.in
+++ b/packaging/RPM/ctdb.spec.in
@@ -145,6 +145,8 @@ development libraries for ctdb
 %{_libdir}/libctdb.a
 
 %changelog
+* Tue Apr 3 2012 : Version 1.41
+ - add back the rebalanceip command
 * Thu Mar 1 2012 : Version 1.13
  - This is the new stable branch for modern features for ctdb.
Main new features are performance/scaling improvements for
diff --git a/tools/ctdb.c b/tools/ctdb.c
index d482eab..a56d579 100644
--- a/tools/ctdb.c
+++ b/tools/ctdb.c
@@ -1619,6 +1619,80 @@ static int control_rebalancenode(struct ctdb_context 
*ctdb, int argc, const char
 }
 
 
+static int rebalance_ip(struct ctdb_context *ctdb, ctdb_sock_addr *addr)
+{
+   struct ctdb_public_ip ip;
+   int ret;
+   uint32_t *nodes;
+   uint32_t disable_time;
+   TDB_DATA data;
+   struct ctdb_node_map *nodemap=NULL;
+   TALLOC_CTX *tmp_ctx = talloc_new(ctdb);
+
+   disable_time = 30;
+   data.dptr  = (uint8_t*)disable_time;
+   data.dsize = sizeof(disable_time);
+   ret = ctdb_client_send_message(ctdb, CTDB_BROADCAST_CONNECTED, 
CTDB_SRVID_DISABLE_IP_CHECK, data);
+   if (ret != 0) {
+   DEBUG(DEBUG_ERR,(Failed to send message to disable 
ipcheck\n));
+   return -1;
+   }
+
+   ip.pnn  = -1;
+   ip.addr = *addr;
+
+   data.dptr  = (uint8_t *)ip;
+   data.dsize = sizeof(ip);
+
+   ret = ctdb_ctrl_getnodemap(ctdb, TIMELIMIT(), options.pnn, tmp_ctx, 
nodemap);
+   if (ret != 0) {
+   DEBUG(DEBUG_ERR, (Unable to get nodemap from node %u\n, 
options.pnn));
+   talloc_free(tmp_ctx);
+   return ret;
+   }
+
+   nodes = list_of_active_nodes(ctdb, nodemap, tmp_ctx, true);
+   ret = ctdb_client_async_control(ctdb, CTDB_CONTROL_RELEASE_IP,
+   nodes, 0,
+   LONGTIMELIMIT(),
+   false, data,
+   NULL, NULL,
+   NULL);
+   if (ret != 0) {
+   DEBUG(DEBUG_ERR,(Failed to release IP on nodes\n));
+   talloc_free(tmp_ctx);
+   return -1;
+   }
+
+   talloc_free(tmp_ctx);
+   return 0;
+}
+
+/*
+  release an ip form all nodes and have it re-assigned by recd
+ */
+static int control_rebalanceip(struct ctdb_context *ctdb, int argc, const char 
**argv)
+{
+   ctdb_sock_addr addr;
+
+   if (argc  1) {
+   usage();
+   return -1;
+   }
+
+   if (parse_ip(argv[0], NULL, 0, addr) == 0) {
+   DEBUG(DEBUG_ERR,(Wrongly formed ip address '%s'\n, argv[0]));
+   return -1;
+   }
+
+   if (rebalance_ip(ctdb, addr) != 0) {
+   DEBUG(DEBUG_ERR,(Error when trying to reassign ip\n));
+   return -1;
+   }
+
+   return 0;
+}
+
 static int getips_store_callback(void *param, void *data)
 {
struct ctdb_public_ip *node_ip = (struct ctdb_public_ip *)data;
@@ -5582,6 +5656,7 @@ static const struct {
{ listnodes,   control_listnodes, false,  true, list all 
nodes in the cluster},
{ reloadnodes, control_reload_nodes_file, false,  false, reload 
the nodes file and restart the transport on all nodes},
{ moveip,  control_moveip,false,  false, 
move/failover an ip address to another node, ip node},
+   { rebalanceip, control_rebalanceip,   false,  false, release 
an ip from the node and let recd rebalance it, ip},
{ addip,   control_addip, true,   false, add a 
ip address to a node, ip/mask iface},
{ delip,   control_delip, false,  false, delete 
an ip address from a node, ip},
{ eventscript, control_eventscript,   true,   false, run the 
eventscript

[SCM] CTDB repository - branch master updated - ctdb-1.13-61-g83aea64

2012-04-02 Thread Ronnie Sahlberg
The branch, master has been updated
   via  83aea6488002aa75dd25752fa0d164afb8b4f300 (commit)
   via  a2ba2c4ec153c8844cf7f5b0de03b2269ec892f3 (commit)
   via  98ac99c4a79fe2ee024890bb27c3ca68dc02d434 (commit)
   via  10ab88a5c822ac403a70a9cd1a6c81020eb91d08 (commit)
   via  2ade83de4e1f318c91489ec9a7f90d1b26f77e1b (commit)
   via  94f1e02406c353c3a0235685acdd107ee5a7f285 (commit)
   via  0cbe7a4f92795203f022252a50a240b8bd4238c3 (commit)
   via  36e690faab3d76919780c8550fe6d49fb6bbfba5 (commit)
   via  45773dab630ee44b39df0dd4b50f43fb1e7bb0ba (commit)
   via  a9e9444a7a78f5ef0e929a8dbd8ca0c68b40acd4 (commit)
  from  00cd75595685dae829758abf1a4cb644af7ed50e (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=master


- Log -
commit 83aea6488002aa75dd25752fa0d164afb8b4f300
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Tue Apr 3 13:43:21 2012 +1000

CTDB: add back the rebalanceip command

commit a2ba2c4ec153c8844cf7f5b0de03b2269ec892f3
Author: Amitay Isaacs ami...@gmail.com
Date:   Fri Mar 30 12:06:05 2012 +1100

build: Substitute POPT macros once and reuse variables

Signed-off-by: Amitay Isaacs ami...@gmail.com

commit 98ac99c4a79fe2ee024890bb27c3ca68dc02d434
Author: Amitay Isaacs ami...@gmail.com
Date:   Tue Mar 27 10:53:25 2012 +1100

recovery: Add prototypes for tdb internal functions

Signed-off-by: Amitay Isaacs ami...@gmail.com

commit 10ab88a5c822ac403a70a9cd1a6c81020eb91d08
Author: Amitay Isaacs ami...@gmail.com
Date:   Wed Mar 21 14:45:56 2012 +1100

build: Use system tdb library if available

Signed-off-by: Amitay Isaacs ami...@gmail.com

commit 2ade83de4e1f318c91489ec9a7f90d1b26f77e1b
Author: Amitay Isaacs ami...@gmail.com
Date:   Wed Mar 21 15:56:45 2012 +1100

build: Use system tevent library if available

Signed-off-by: Amitay Isaacs ami...@gmail.com

commit 94f1e02406c353c3a0235685acdd107ee5a7f285
Author: Amitay Isaacs ami...@gmail.com
Date:   Wed Mar 21 13:18:10 2012 +1100

build: Use system talloc library if available

Signed-off-by: Amitay Isaacs ami...@gmail.com

commit 0cbe7a4f92795203f022252a50a240b8bd4238c3
Author: Amitay Isaacs ami...@gmail.com
Date:   Wed Mar 21 16:08:57 2012 +1100

build: Display correct LIB_FLAGS while building

Signed-off-by: Amitay Isaacs ami...@gmail.com

commit 36e690faab3d76919780c8550fe6d49fb6bbfba5
Author: Amitay Isaacs ami...@gmail.com
Date:   Tue Mar 20 09:22:54 2012 +1100

build: Remove re-definition of same variable

Signed-off-by: Amitay Isaacs ami...@gmail.com

commit 45773dab630ee44b39df0dd4b50f43fb1e7bb0ba
Author: Amitay Isaacs ami...@gmail.com
Date:   Mon Feb 6 16:45:50 2012 +1100

packaging: Setup directories for rpmbuild

Signed-off-by: Amitay Isaacs ami...@gmail.com

commit a9e9444a7a78f5ef0e929a8dbd8ca0c68b40acd4
Author: Amitay Isaacs ami...@gmail.com
Date:   Wed Jan 11 19:04:39 2012 +1100

build: Add rules to create ctags/etags

Signed-off-by: Amitay Isaacs ami...@gmail.com

---

Summary of changes:
 Makefile.in |   44 ++---
 lib/talloc/libtalloc.m4 |   63 +---
 lib/tdb/libtdb.m4   |   66 +
 lib/tevent/libtevent.m4 |   66 --
 packaging/RPM/makerpms.sh   |8 
 server/ctdb_recover.c   |7 
 server/ctdb_update_record.c |8 ++--
 tools/ctdb.c|   75 +++
 8 files changed, 256 insertions(+), 81 deletions(-)


Changeset truncated at 500 lines:

diff --git a/Makefile.in b/Makefile.in
index 90d7b08..1794be5 100755
--- a/Makefile.in
+++ b/Makefile.in
@@ -29,6 +29,18 @@ POPT_LIBS = @POPT_LIBS@
 POPT_CFLAGS = @POPT_CFLAGS@
 POPT_OBJ = @POPT_OBJ@
 
+TALLOC_LIBS = @TALLOC_LIBS@
+TALLOC_CFLAGS = @TALLOC_CFLAGS@
+TALLOC_OBJ = @TALLOC_OBJ@
+
+TEVENT_LIBS = @TEVENT_LIBS@
+TEVENT_CFLAGS = @TEVENT_CFLAGS@
+TEVENT_OBJ = @TEVENT_OBJ@
+
+TDB_LIBS = @TDB_LIBS@
+TDB_CFLAGS = @TDB_CFLAGS@
+TDB_OBJ = @TDB_OBJ@
+
 SOCKET_WRAPPER_OBJ = @SOCKET_WRAPPER_OBJS@
 
 PMDA_LIBS = -lpcp -lpcp_pmda
@@ -36,7 +48,7 @@ PMDA_INSTALL = @CTDB_PMDA_INSTALL@
 PMDA_DEST_DIR = /var/lib/pcp/pmdas
 
 CFLAGS=@CPPFLAGS@ -g -I$(srcdir)/include -Iinclude -Ilib -Ilib/util 
-I$(srcdir) \
-   -I@tallocdir@ -I@tdbdir@/include -I@libreplacedir@ \
+   $(TALLOC_CFLAGS) $(TEVENT_CFLAGS) $(TDB_CFLAGS) -I@libreplacedir@ \
-DVARDIR=\$(localstatedir)\ -DETCDIR=\$(etcdir)\ \
-DLOGDIR=\$(logdir)\ \
-DUSE_MMAP=1 -DTEVENT_DEPRECATED_QUIET=1 @CFLAGS@ 
-Wno-format-zero-length $(POPT_CFLAGS) \
@@ -46,7 +58,8 @@ LDSHFLAGS=-fPIC -shared
 #LDSHFLAGS=-fPIC -shared -Wl,-Bsymbolic -Wl,-z,relo -Wl,-Bsymbolic-funtions 
-Wl,--as-needed -Wl,-z,defs
 SHLD=${CC

[SCM] CTDB repository - branch 1.13 updated - ctdb-1.13-68-gaaec462

2012-03-28 Thread Ronnie Sahlberg
The branch, 1.13 has been updated
   via  aaec46211b72f7d6886832301eee169d4b6a5f80 (commit)
  from  c4aa79c2a89d362fe54c6bd86dc1831236047cfc (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=1.13


- Log -
commit aaec46211b72f7d6886832301eee169d4b6a5f80
Author: Martin Schwenke mar...@meltin.net
Date:   Wed Mar 28 14:50:36 2012 +1100

Initscript - add backup of corrupt non-persistent databases

Corrupt non-persistent databases never get analysed because ctdbd
zeroes them at startup.

Modify the initscript so that corrupt non-persistent databases are
moved aside to a backup.  If the number of backups for a particular
database exceeds $CTDB_MAX_CORRUPT_DB_BACKUPS (default 10) then the
oldest excess backups are garbage collected.

Abstracts from and cleans up the code for checking persistent
databases.

Logging of related messages is done to syslog or a log file as
specified.

Signed-off-by: Martin Schwenke mar...@meltin.net

---

Summary of changes:
 config/ctdb.init |  156 --
 1 files changed, 93 insertions(+), 63 deletions(-)


Changeset truncated at 500 lines:

diff --git a/config/ctdb.init b/config/ctdb.init
index 0b86344..0a4d29b 100755
--- a/config/ctdb.init
+++ b/config/ctdb.init
@@ -107,85 +107,112 @@ build_ctdb_options () {
 maybe_set --max-persistent-check-errors 
$CTDB_MAX_PERSISTENT_CHECK_ERRORS
 }
 
-check_tdb () {
-   local PDBASE=$1
-
-   test x$TDBTOOL_HAS_CHECK = x1  {
-   #
-   # Note tdbtool always exits with 0
-   #
-   local OK=`tdbtool $PDBASE check | grep Database integrity is 
OK | wc -l`
-   test x$OK = x1 || {
-   return 1;
-   }
-
-   return 0;
-   }
-
-   tdbdump $PDBASE /dev/null 2/dev/null || {
-   return $?;
-   }
-
-   return 0;
-}
-
-check_persistent_databases () {
-PERSISTENT_DB_DIR=${CTDB_DBDIR:-/var/ctdb}/persistent
-mkdir -p $PERSISTENT_DB_DIR 2/dev/null
-local ERRCOUNT=$CTDB_MAX_PERSISTENT_CHECK_ERRORS
+# Log given message or stdin to either syslog or a CTDB log file
+do_log ()
+{
+if [ $CTDB_SYSLOG = yes -o \
+   ${CTDB_OPTIONS#*--syslog} != $CTDB_OPTIONS ] ; then
 
-test -z $ERRCOUNT  {
-   ERRCOUNT=0
-}
-test x$ERRCOUNT != x0  {
-   return 0;
-}
-
-if test -x /usr/bin/tdbtool ; then
-HAVE_TDBTOOL=1
+   logger -t ctdb.init $@
 else
-HAVE_TDBTOOL=0
+   _l=${CTDB_LOGFILE:-/var/log/log.ctdb}
+   {
+   date
+   if [ -n $* ] ; then
+   echo $*
+   else
+   cat
+   fi
+   } $_l
 fi
+}
 
-if test x$HAVE_TDBTOOL = x1 ; then
-TDBTOOL_HAS_CHECK=`echo help | /usr/bin/tdbtool | grep check | wc -l`
+select_tdb_checker ()
+{
+# Find the best TDB consistency check available.
+use_tdb_tool_check=false
+if [ -x /usr/bin/tdbtool ]  \
+   echo help | /usr/bin/tdbtool | grep -q check ; then
+
+   use_tdb_tool_check=true
+elif [ -x /usr/bin/tdbtool -a -x /usr/bin/tdbdump ] ; then
+   do_log EOF
+WARNING: The installed 'tdbtool' does not offer the 'check' subcommand.
+ Using 'tdbdump' for database checks.
+ Consider updating 'tdbtool' for better checks!
+EOF
+elif [ -x /usr/bin/tdbdump ] ; then
+   do_log EOF
+WARNING: 'tdbtool' is not available.
+ Using 'tdbdump' to check the databases.
+ Consider installing a recent 'tdbtool' for better checks!
+EOF
 else
-TDBTOOL_HAS_CHECK=0
+   do_log EOF
+WARNING: Cannot check databases since neither
+ 'tdbdump' nor 'tdbtool check' is available.
+ Consider installing tdbtool or at least tdbdump!
+EOF
+return 1
 fi
+}
+
+check_tdb ()
+{
+_db=$1
 
-if test -x /usr/bin/tdbdump ; then
-HAVE_TDBDUMP=1
+if $use_tdb_tool_check ; then
+   # tdbtool always exits with 0  :-(
+   if tdbtool $_db check 2/dev/null |
+   grep -q Database integrity is OK ; then
+   return 0
+   else
+   return 1
+   fi
 else
-HAVE_TDBDUMP=0
+   tdbdump $_db /dev/null 2/dev/null
+   return $?
 fi
+}
 
-if test x$HAVE_TDBDUMP = x0 -a x$TDBTOOL_HAS_CHECK = x0 ; then
-echo WARNING: Cannot check persistent databases since
-echo neither 'tdbdump' nor 'tdbtool check' is available.
-echo Consider installing tdbtool or at least tdbdump!
-return 0
-fi
+check_persistent_databases ()
+{
+_dir=${CTDB_DBDIR_PERSISTENT:-${CTDB_DBDIR:-/var/ctdb}/persistent}
+mkdir -p $_dir 2/dev/null
 
-if test x$HAVE_TDBDUMP = x1 -a x$TDBTOOL_HAS_CHECK = x0 ; then
-if test x$HAVE_TDBTOOL = x0; then
-echo WARNING: 'tdbtool' is not 

[SCM] CTDB repository - branch 1.2.40 updated - ctdb-1.9.1-559-gb54938c

2012-03-28 Thread Ronnie Sahlberg
The branch, 1.2.40 has been updated
   via  b54938c8dfaac046aee9eec92724e7ba2153d65a (commit)
   via  ba78c020dcb525762e5fde943fc60084120884ed (commit)
  from  d967d1bcf72d32312ca765bb467d03ddd4690cb7 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=1.2.40


- Log -
commit b54938c8dfaac046aee9eec92724e7ba2153d65a
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Thu Mar 29 10:10:04 2012 +1100

New version 1.2.43

commit ba78c020dcb525762e5fde943fc60084120884ed
Author: Volker Lendecke v...@samba.org
Date:   Mon Oct 31 13:29:13 2011 +0100

Add CTDB_CONTROL_CHECK_SRVID

---

Summary of changes:
 include/ctdb_private.h |2 ++
 include/ctdb_protocol.h|1 +
 packaging/RPM/ctdb.spec.in |4 +++-
 server/ctdb_control.c  |3 +++
 server/ctdb_daemon.c   |   36 
 5 files changed, 45 insertions(+), 1 deletions(-)


Changeset truncated at 500 lines:

diff --git a/include/ctdb_private.h b/include/ctdb_private.h
index 8180722..e3b5a20 100644
--- a/include/ctdb_private.h
+++ b/include/ctdb_private.h
@@ -967,6 +967,8 @@ int ctdb_dispatch_message(struct ctdb_context *ctdb, 
uint64_t srvid, TDB_DATA da
 int daemon_register_message_handler(struct ctdb_context *ctdb, uint32_t 
client_id, uint64_t srvid);
 int ctdb_deregister_message_handler(struct ctdb_context *ctdb, uint64_t srvid, 
void *private_data);
 int daemon_deregister_message_handler(struct ctdb_context *ctdb, uint32_t 
client_id, uint64_t srvid);
+int daemon_check_srvids(struct ctdb_context *ctdb, TDB_DATA indata,
+   TDB_DATA *outdata);
 
 int32_t ctdb_ltdb_enable_seqnum(struct ctdb_context *ctdb, uint32_t db_id);
 int32_t ctdb_ltdb_update_seqnum(struct ctdb_context *ctdb, uint32_t db_id, 
uint32_t srcnode);
diff --git a/include/ctdb_protocol.h b/include/ctdb_protocol.h
index c874148..99acd43 100644
--- a/include/ctdb_protocol.h
+++ b/include/ctdb_protocol.h
@@ -372,6 +372,7 @@ enum ctdb_controls {CTDB_CONTROL_PROCESS_EXISTS  = 
0,
CTDB_CONTROL_GET_STAT_HISTORY= 127,
CTDB_CONTROL_SCHEDULE_FOR_DELETION   = 128,
CTDB_CONTROL_SET_DB_READONLY = 129,
+   CTDB_CONTROL_CHECK_SRVIDS= 130,
 };
 
 /*
diff --git a/packaging/RPM/ctdb.spec.in b/packaging/RPM/ctdb.spec.in
index 7519be1..9ec9192 100644
--- a/packaging/RPM/ctdb.spec.in
+++ b/packaging/RPM/ctdb.spec.in
@@ -3,7 +3,7 @@ Name: ctdb
 Summary: Clustered TDB
 Vendor: Samba Team
 Packager: Samba Team sa...@samba.org
-Version: 1.2.42
+Version: 1.2.43
 Release: 1GITHASH
 Epoch: 0
 License: GNU GPL version 3
@@ -144,6 +144,8 @@ development libraries for ctdb
 %{_libdir}/libctdb.a
 
 %changelog
+* Thu Mar 29 2012 : Version 1.2.43
+ - Add CTDB_CONTROL_CHECK_SRVID control
 * Mon Mar 12 2012 : Version 1.2.42
  - Allow non-readonly database UPDATE_RECORD to create non-existing records
or else net command and the registry breaks.
diff --git a/server/ctdb_control.c b/server/ctdb_control.c
index 9c2f742..5968b3e 100644
--- a/server/ctdb_control.c
+++ b/server/ctdb_control.c
@@ -263,6 +263,9 @@ static int32_t ctdb_control_dispatch(struct ctdb_context 
*ctdb,
case CTDB_CONTROL_DEREGISTER_SRVID:
return daemon_deregister_message_handler(ctdb, client_id, 
srvid);
 
+   case CTDB_CONTROL_CHECK_SRVIDS:
+   return daemon_check_srvids(ctdb, indata, outdata);
+
case CTDB_CONTROL_ENABLE_SEQNUM:
CHECK_CONTROL_DATA_SIZE(sizeof(uint32_t));
return ctdb_ltdb_enable_seqnum(ctdb, *(uint32_t *)indata.dptr);
diff --git a/server/ctdb_daemon.c b/server/ctdb_daemon.c
index caa70cf..81cec67 100644
--- a/server/ctdb_daemon.c
+++ b/server/ctdb_daemon.c
@@ -204,6 +204,42 @@ int daemon_deregister_message_handler(struct ctdb_context 
*ctdb, uint32_t client
return ctdb_deregister_message_handler(ctdb, srvid, client);
 }
 
+int daemon_check_srvids(struct ctdb_context *ctdb, TDB_DATA indata,
+   TDB_DATA *outdata)
+{
+   uint64_t *ids;
+   int i, num_ids;
+   uint8_t *results;
+
+   if ((indata.dsize % sizeof(uint64_t)) != 0) {
+   DEBUG(DEBUG_ERR, (Bad indata in daemon_check_srvids, 
+ size=%d\n, (int)indata.dsize));
+   return -1;
+   }
+
+   ids = (uint64_t *)indata.dptr;
+   num_ids = indata.dsize / 8;
+
+   results = talloc_zero_array(outdata, uint8_t, (num_ids+7)/8);
+   if (results == NULL) {
+   DEBUG(DEBUG_ERR, (talloc failed in daemon_check_srvids\n));
+   return -1;
+   }
+   for (i=0; inum_ids; i++) {
+   struct ctdb_message_list *ml;
+   for (ml=ctdb-message_list; ml; ml=ml-next) {
+   if (ml-srvid == ids[i

[SCM] CTDB repository - branch master updated - ctdb-1.13-51-g00cd755

2012-03-28 Thread Ronnie Sahlberg
The branch, master has been updated
   via  00cd75595685dae829758abf1a4cb644af7ed50e (commit)
  from  5cdeb0dd8827b13b06265c75dfac359bcd6ad10c (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=master


- Log -
commit 00cd75595685dae829758abf1a4cb644af7ed50e
Author: Martin Schwenke mar...@meltin.net
Date:   Wed Mar 28 14:50:36 2012 +1100

Initscript - add backup of corrupt non-persistent databases

Corrupt non-persistent databases never get analysed because ctdbd
zeroes them at startup.

Modify the initscript so that corrupt non-persistent databases are
moved aside to a backup.  If the number of backups for a particular
database exceeds $CTDB_MAX_CORRUPT_DB_BACKUPS (default 10) then the
oldest excess backups are garbage collected.

Abstracts from and cleans up the code for checking persistent
databases.

Logging of related messages is done to syslog or a log file as
specified.

Signed-off-by: Martin Schwenke mar...@meltin.net

---

Summary of changes:
 config/ctdb.init |  156 --
 1 files changed, 93 insertions(+), 63 deletions(-)


Changeset truncated at 500 lines:

diff --git a/config/ctdb.init b/config/ctdb.init
index 0b86344..0a4d29b 100755
--- a/config/ctdb.init
+++ b/config/ctdb.init
@@ -107,85 +107,112 @@ build_ctdb_options () {
 maybe_set --max-persistent-check-errors 
$CTDB_MAX_PERSISTENT_CHECK_ERRORS
 }
 
-check_tdb () {
-   local PDBASE=$1
-
-   test x$TDBTOOL_HAS_CHECK = x1  {
-   #
-   # Note tdbtool always exits with 0
-   #
-   local OK=`tdbtool $PDBASE check | grep Database integrity is 
OK | wc -l`
-   test x$OK = x1 || {
-   return 1;
-   }
-
-   return 0;
-   }
-
-   tdbdump $PDBASE /dev/null 2/dev/null || {
-   return $?;
-   }
-
-   return 0;
-}
-
-check_persistent_databases () {
-PERSISTENT_DB_DIR=${CTDB_DBDIR:-/var/ctdb}/persistent
-mkdir -p $PERSISTENT_DB_DIR 2/dev/null
-local ERRCOUNT=$CTDB_MAX_PERSISTENT_CHECK_ERRORS
+# Log given message or stdin to either syslog or a CTDB log file
+do_log ()
+{
+if [ $CTDB_SYSLOG = yes -o \
+   ${CTDB_OPTIONS#*--syslog} != $CTDB_OPTIONS ] ; then
 
-test -z $ERRCOUNT  {
-   ERRCOUNT=0
-}
-test x$ERRCOUNT != x0  {
-   return 0;
-}
-
-if test -x /usr/bin/tdbtool ; then
-HAVE_TDBTOOL=1
+   logger -t ctdb.init $@
 else
-HAVE_TDBTOOL=0
+   _l=${CTDB_LOGFILE:-/var/log/log.ctdb}
+   {
+   date
+   if [ -n $* ] ; then
+   echo $*
+   else
+   cat
+   fi
+   } $_l
 fi
+}
 
-if test x$HAVE_TDBTOOL = x1 ; then
-TDBTOOL_HAS_CHECK=`echo help | /usr/bin/tdbtool | grep check | wc -l`
+select_tdb_checker ()
+{
+# Find the best TDB consistency check available.
+use_tdb_tool_check=false
+if [ -x /usr/bin/tdbtool ]  \
+   echo help | /usr/bin/tdbtool | grep -q check ; then
+
+   use_tdb_tool_check=true
+elif [ -x /usr/bin/tdbtool -a -x /usr/bin/tdbdump ] ; then
+   do_log EOF
+WARNING: The installed 'tdbtool' does not offer the 'check' subcommand.
+ Using 'tdbdump' for database checks.
+ Consider updating 'tdbtool' for better checks!
+EOF
+elif [ -x /usr/bin/tdbdump ] ; then
+   do_log EOF
+WARNING: 'tdbtool' is not available.
+ Using 'tdbdump' to check the databases.
+ Consider installing a recent 'tdbtool' for better checks!
+EOF
 else
-TDBTOOL_HAS_CHECK=0
+   do_log EOF
+WARNING: Cannot check databases since neither
+ 'tdbdump' nor 'tdbtool check' is available.
+ Consider installing tdbtool or at least tdbdump!
+EOF
+return 1
 fi
+}
+
+check_tdb ()
+{
+_db=$1
 
-if test -x /usr/bin/tdbdump ; then
-HAVE_TDBDUMP=1
+if $use_tdb_tool_check ; then
+   # tdbtool always exits with 0  :-(
+   if tdbtool $_db check 2/dev/null |
+   grep -q Database integrity is OK ; then
+   return 0
+   else
+   return 1
+   fi
 else
-HAVE_TDBDUMP=0
+   tdbdump $_db /dev/null 2/dev/null
+   return $?
 fi
+}
 
-if test x$HAVE_TDBDUMP = x0 -a x$TDBTOOL_HAS_CHECK = x0 ; then
-echo WARNING: Cannot check persistent databases since
-echo neither 'tdbdump' nor 'tdbtool check' is available.
-echo Consider installing tdbtool or at least tdbdump!
-return 0
-fi
+check_persistent_databases ()
+{
+_dir=${CTDB_DBDIR_PERSISTENT:-${CTDB_DBDIR:-/var/ctdb}/persistent}
+mkdir -p $_dir 2/dev/null
 
-if test x$HAVE_TDBDUMP = x1 -a x$TDBTOOL_HAS_CHECK = x0 ; then
-if test x$HAVE_TDBTOOL = x0; then
-echo WARNING: 'tdbtool' is not 

[SCM] CTDB repository - branch 1.13 updated - ctdb-1.13-67-gc4aa79c

2012-03-22 Thread Ronnie Sahlberg
The branch, 1.13 has been updated
   via  c4aa79c2a89d362fe54c6bd86dc1831236047cfc (commit)
  from  41ee3cb65b63605ab968cf1eef05a75256d4db6c (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=1.13


- Log -
commit c4aa79c2a89d362fe54c6bd86dc1831236047cfc
Author: Martin Schwenke mar...@meltin.net
Date:   Fri Mar 23 14:31:31 2012 +1100

Add policy routing documentation to ctdbd(1).

Signed-off-by: Martin Schwenke mar...@meltin.net

---

Summary of changes:
 doc/ctdbd.1  |  205 -
 doc/ctdbd.1.html |  274 --
 doc/ctdbd.1.xml  |  230 +
 3 files changed, 635 insertions(+), 74 deletions(-)


Changeset truncated at 500 lines:

diff --git a/doc/ctdbd.1 b/doc/ctdbd.1
index 7e5443e..be527aa 100644
--- a/doc/ctdbd.1
+++ b/doc/ctdbd.1
@@ -1,13 +1,13 @@
 '\ t
 .\ Title: ctdbd
 .\Author: [FIXME: author] [see http://docbook.sf.net/el/author]
-.\ Generator: DocBook XSL Stylesheets v1.75.2 http://docbook.sf.net/
-.\  Date: 03/22/2012
+.\ Generator: DocBook XSL Stylesheets v1.76.1 http://docbook.sf.net/
+.\  Date: 03/23/2012
 .\Manual: CTDB - clustered TDB database
 .\Source: ctdb
 .\  Language: English
 .\
-.TH CTDBD 1 03/22/2012 ctdb CTDB \- clustered TDB database
+.TH CTDBD 1 03/23/2012 ctdb CTDB \- clustered TDB database
 .\ -
 .\ * Define some portability stuff
 .\ -
@@ -785,6 +785,205 @@ If you want to change the NATGW configuration on a node :
 .if n \{\
 .RE
 .\}
+.SH POLICY ROUTING
+.PP
+A node running CTDB may be a component of a complex network topology\. In 
particular, public addresses may be spread across several different networks 
(or VLANs) and it may not be possible to route packets from these public 
addresses via the system\*(Aqs default route\. Therefore, CTDB has support for 
policy routing via the 13\.per_ip_routing eventscript\. This allows routing 
to be specified for packets sourced from each public address\. The routes are 
added and removed as CTDB moves public addresses between nodes\.
+.SS Configuration variables
+.PP
+There are 4 configuration variables related to policy routing:
+.PP
+\fICTDB_PER_IP_ROUTING_CONF\fR
+.RS 4
+The name of a configuration file that specifies the desired routes for each 
source address\. The configuration file format is discussed below\. A 
recommended value is
+/etc/ctdb/policy_routing\.
+.sp
+The special value
+\fB__auto_link_local__\fR
+indicates that no configuration file is provided and that CTDB should generate 
reasonable link\-local routes for each public address\.
+.RE
+.PP
+\fICTDB_PER_IP_ROUTING_RULE_PREF\fR
+.RS 4
+This sets the priority (or preference) for the routing rules that are added by 
CTDB\.
+.sp
+This should be (strictly) greater than 0 and (strictly) less than 32766\. A 
priority of 100 is recommended, unless this conflicts with a priority already 
in use on the system\. See ip(8) for more details\.
+.RE
+.PP
+\fICTDB_PER_IP_ROUTING_TABLE_ID_LOW\fR, \fICTDB_PER_IP_ROUTING_TABLE_ID_HIGH\fR
+.RS 4
+CTDB determines a unique routing table number to use for the routing related 
to each public address\. These variables indicate the minimum and maximum 
routing table numbers that are used\.
+.sp
+The ip command uses some reserved routing table numbers below 255\. 
Therefore, CTDB_PER_IP_ROUTING_TABLE_ID_LOW should be (strictly) greater than 
255\. 1000 and 9000 are recommended values, unless this range conflicts with 
routing tables numbers already in use on the system\.
+.sp
+CTDB uses the standard file
+/etc/iproute2/rt_tables
+to maintain a mapping between the routing table numbers and labels\. The 
label for a public address addr;gt; will look like ctdb\.addr\. This means 
that the associated rules and routes are easy to read (and manipulate)\.
+.RE
+.SS Configuration file
+.PP
+The format of each line is:
+.sp
+.if n \{\
+.RS 4
+.\}
+.nf
+public_address network [ gateway ]
+  
+.fi
+.if n \{\
+.RE
+.\}
+.PP
+Leading whitespace is ignored and arbitrary whitespace may be used as a 
separator\. Lines that have a public address item that doesn\*(Aqt match an 
actual public address are ignored\. This means that comment lines can be added 
using a leading character such as \*(Aq#\*(Aq, since this will never match an 
IP address\.
+.PP
+A line without a gateway indicates a link local route\.
+.PP
+For example, consider the configuration line:
+.sp
+.if n \{\
+.RS 4
+.\}
+.nf
+  192\.168\.1\.99   192\.168\.1\.1/24
+  
+.fi
+.if n \{\
+.RE
+.\}
+.PP
+If the corresponding public_addresses line is:
+.sp
+.if n \{\
+.RS 4
+.\}
+.nf
+  192\.168\.1\.99/24 eth2,eth3
+  
+.fi
+.if n \{\
+.RE
+.\}

[SCM] CTDB repository - branch master updated - ctdb-1.13-50-g5cdeb0d

2012-03-22 Thread Ronnie Sahlberg
The branch, master has been updated
   via  5cdeb0dd8827b13b06265c75dfac359bcd6ad10c (commit)
  from  7293762046e25a9c96e6417e34e18c214ebc6e55 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=master


- Log -
commit 5cdeb0dd8827b13b06265c75dfac359bcd6ad10c
Author: Martin Schwenke mar...@meltin.net
Date:   Fri Mar 23 14:31:31 2012 +1100

Add policy routing documentation to ctdbd(1).

Signed-off-by: Martin Schwenke mar...@meltin.net

---

Summary of changes:
 doc/ctdbd.1  |  205 -
 doc/ctdbd.1.html |  274 --
 doc/ctdbd.1.xml  |  230 +
 3 files changed, 635 insertions(+), 74 deletions(-)


Changeset truncated at 500 lines:

diff --git a/doc/ctdbd.1 b/doc/ctdbd.1
index 7e5443e..be527aa 100644
--- a/doc/ctdbd.1
+++ b/doc/ctdbd.1
@@ -1,13 +1,13 @@
 '\ t
 .\ Title: ctdbd
 .\Author: [FIXME: author] [see http://docbook.sf.net/el/author]
-.\ Generator: DocBook XSL Stylesheets v1.75.2 http://docbook.sf.net/
-.\  Date: 03/22/2012
+.\ Generator: DocBook XSL Stylesheets v1.76.1 http://docbook.sf.net/
+.\  Date: 03/23/2012
 .\Manual: CTDB - clustered TDB database
 .\Source: ctdb
 .\  Language: English
 .\
-.TH CTDBD 1 03/22/2012 ctdb CTDB \- clustered TDB database
+.TH CTDBD 1 03/23/2012 ctdb CTDB \- clustered TDB database
 .\ -
 .\ * Define some portability stuff
 .\ -
@@ -785,6 +785,205 @@ If you want to change the NATGW configuration on a node :
 .if n \{\
 .RE
 .\}
+.SH POLICY ROUTING
+.PP
+A node running CTDB may be a component of a complex network topology\. In 
particular, public addresses may be spread across several different networks 
(or VLANs) and it may not be possible to route packets from these public 
addresses via the system\*(Aqs default route\. Therefore, CTDB has support for 
policy routing via the 13\.per_ip_routing eventscript\. This allows routing 
to be specified for packets sourced from each public address\. The routes are 
added and removed as CTDB moves public addresses between nodes\.
+.SS Configuration variables
+.PP
+There are 4 configuration variables related to policy routing:
+.PP
+\fICTDB_PER_IP_ROUTING_CONF\fR
+.RS 4
+The name of a configuration file that specifies the desired routes for each 
source address\. The configuration file format is discussed below\. A 
recommended value is
+/etc/ctdb/policy_routing\.
+.sp
+The special value
+\fB__auto_link_local__\fR
+indicates that no configuration file is provided and that CTDB should generate 
reasonable link\-local routes for each public address\.
+.RE
+.PP
+\fICTDB_PER_IP_ROUTING_RULE_PREF\fR
+.RS 4
+This sets the priority (or preference) for the routing rules that are added by 
CTDB\.
+.sp
+This should be (strictly) greater than 0 and (strictly) less than 32766\. A 
priority of 100 is recommended, unless this conflicts with a priority already 
in use on the system\. See ip(8) for more details\.
+.RE
+.PP
+\fICTDB_PER_IP_ROUTING_TABLE_ID_LOW\fR, \fICTDB_PER_IP_ROUTING_TABLE_ID_HIGH\fR
+.RS 4
+CTDB determines a unique routing table number to use for the routing related 
to each public address\. These variables indicate the minimum and maximum 
routing table numbers that are used\.
+.sp
+The ip command uses some reserved routing table numbers below 255\. 
Therefore, CTDB_PER_IP_ROUTING_TABLE_ID_LOW should be (strictly) greater than 
255\. 1000 and 9000 are recommended values, unless this range conflicts with 
routing tables numbers already in use on the system\.
+.sp
+CTDB uses the standard file
+/etc/iproute2/rt_tables
+to maintain a mapping between the routing table numbers and labels\. The 
label for a public address addr;gt; will look like ctdb\.addr\. This means 
that the associated rules and routes are easy to read (and manipulate)\.
+.RE
+.SS Configuration file
+.PP
+The format of each line is:
+.sp
+.if n \{\
+.RS 4
+.\}
+.nf
+public_address network [ gateway ]
+  
+.fi
+.if n \{\
+.RE
+.\}
+.PP
+Leading whitespace is ignored and arbitrary whitespace may be used as a 
separator\. Lines that have a public address item that doesn\*(Aqt match an 
actual public address are ignored\. This means that comment lines can be added 
using a leading character such as \*(Aq#\*(Aq, since this will never match an 
IP address\.
+.PP
+A line without a gateway indicates a link local route\.
+.PP
+For example, consider the configuration line:
+.sp
+.if n \{\
+.RS 4
+.\}
+.nf
+  192\.168\.1\.99   192\.168\.1\.1/24
+  
+.fi
+.if n \{\
+.RE
+.\}
+.PP
+If the corresponding public_addresses line is:
+.sp
+.if n \{\
+.RS 4
+.\}
+.nf
+  192\.168\.1\.99/24 eth2,eth3
+  
+.fi
+.if n \{\
+.RE

[SCM] CTDB repository - branch 1.13 updated - ctdb-1.13-14-gae86409

2012-03-21 Thread Ronnie Sahlberg
The branch, 1.13 has been updated
   via  ae86409ded3a242dd2f7452287ebdb55e006b055 (commit)
   via  980beee0aea319d8670a538858895ba662e036f8 (commit)
  from  6802b43c09cd11cc3074882e7a43fae52dc914a5 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=1.13


- Log -
commit ae86409ded3a242dd2f7452287ebdb55e006b055
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Thu Mar 22 09:05:49 2012 +1100

Make NoIPFailback a node local setting. Nodes that have NoIPFailback set to 
!0 can not takeover new ip addresses during failover.

Remove the old global setting for this unused tunable and add it as a new 
node flag. This node flag is only valid/defined within the takeover subsystem 
in the recovery daemon. Add async functions to collec the NoIPFailback settings 
for each node.

This will later e used to disqualify certain nodes from being takeover 
targets when we perform reallocation.

commit 980beee0aea319d8670a538858895ba662e036f8
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Wed Mar 21 16:41:20 2012 +1100

New version 1.13.1

---

Summary of changes:
 include/ctdb_protocol.h |2 +
 server/ctdb_takeover.c  |   65 ++
 2 files changed, 50 insertions(+), 17 deletions(-)


Changeset truncated at 500 lines:

diff --git a/include/ctdb_protocol.h b/include/ctdb_protocol.h
index a153f01..ae989a9 100644
--- a/include/ctdb_protocol.h
+++ b/include/ctdb_protocol.h
@@ -559,6 +559,8 @@ struct ctdb_node_map {
 #define NODE_FLAGS_DISABLED
(NODE_FLAGS_UNHEALTHY|NODE_FLAGS_PERMANENTLY_DISABLED)
 #define NODE_FLAGS_INACTIVE
(NODE_FLAGS_DELETED|NODE_FLAGS_DISCONNECTED|NODE_FLAGS_BANNED|NODE_FLAGS_STOPPED)
 
+#define NODE_FLAGS_NOIPFAILBACK0x0100 /* this node can not 
be failed back onto, this flag is ONLY valid within the recovery daemon */
+
 
 struct ctdb_public_ip {
uint32_t pnn;
diff --git a/server/ctdb_takeover.c b/server/ctdb_takeover.c
index bff22a8..dba375b 100644
--- a/server/ctdb_takeover.c
+++ b/server/ctdb_takeover.c
@@ -1980,19 +1980,6 @@ try_again:
basic_allocate_unassigned(ctdb, nodemap, mask, all_ips);
}
 
-   /* If we dont want ips to fail back after a node becomes healthy
-  again, we wont even try to reallocat the ip addresses so that
-  they are evenly spread out.
-  This can NOT be used at the same time as DeterministicIPs !
-   */
-   if (1 == ctdb-tunable.no_ip_failback) {
-   if (1 == ctdb-tunable.deterministic_public_ips) {
-   DEBUG(DEBUG_ERR, (ERROR: You can not use 
'DeterministicIPs' and 'NoIPFailback' at the same time\n));
-   }
-   goto finished;
-   }
-
-
/* now, try to make sure the ip adresses are evenly distributed
   across the node.
*/
@@ -2008,16 +1995,37 @@ try_again:
 
/* finished distributing the public addresses, now just send the 
   info out to the nodes
-   */
-finished:
-
-   /* at this point -pnn is the node which will own each IP
+  at this point -pnn is the node which will own each IP
   or -1 if there is no node that can cover this ip
*/
 
return;
 }
 
+static void noipfailback_cb(struct ctdb_context *ctdb, uint32_t pnn, int32_t 
res, TDB_DATA outdata, void *callback)
+{
+   struct ctdb_node_map *nodemap = (struct ctdb_node_map *)callback;
+
+   if (res != 0) {
+   DEBUG(DEBUG_ERR,(Failure to read NoIPFailback tunable from 
remote node %d\n, pnn));
+   return;
+   }
+
+   if (outdata.dsize != sizeof(uint32_t)) {
+   DEBUG(DEBUG_ERR,(Wrong size of returned data when reading 
NoIPFailback tunable from node %d. Expected %d bytes but received %d bytes\n, 
pnn, (int)sizeof(uint32_t), (int)outdata.dsize));
+   return;
+   }
+
+   if (pnn = nodemap-num) {
+   DEBUG(DEBUG_ERR,(Got NoIPFailback reply from node %d but 
nodemap only has %d entries\n, pnn, nodemap-num));
+   return;
+   }
+
+   if (*(uint32_t *)outdata.dptr != 0) {
+   nodemap-nodes[pnn].flags |= NODE_FLAGS_NOIPFAILBACK;
+   }
+}
+
 /*
   make any IP alias changes for public addresses that are necessary 
  */
@@ -2026,6 +2034,7 @@ int ctdb_takeover_run(struct ctdb_context *ctdb, struct 
ctdb_node_map *nodemap)
int i;
struct ctdb_public_ip ip;
struct ctdb_public_ipv4 ipv4;
+   struct ctdb_control_get_tunable *t;
uint32_t *nodes;
struct ctdb_public_ip_list *all_ips, *tmp_ip;
TDB_DATA data;
@@ -2042,6 +2051,28 @@ int ctdb_takeover_run(struct ctdb_context *ctdb, struct 
ctdb_node_map *nodemap)
goto ipreallocated;
}
 
+
+   /* assume all nodes do

[SCM] CTDB repository - branch master updated - ctdb-1.13-18-g668f3e8

2012-03-21 Thread Ronnie Sahlberg
The branch, master has been updated
   via  668f3e88a9e5f598706952b7140547640c85a5ed (commit)
  from  0ae2a46e37df8d84ce0c76b7a5d32d6e93af709b (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=master


- Log -
commit 668f3e88a9e5f598706952b7140547640c85a5ed
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Thu Mar 22 09:05:49 2012 +1100

Make NoIPFailback a node local setting. Nodes that have NoIPFailback set to 
!0 can not takeover new ip addresses during failover.

Remove the old global setting for this unused tunable and add it as a new 
node flag. This node flag is only valid/defined within the takeover subsystem 
in the recovery daemon. Add async functions to collec the NoIPFailback settings 
for each node.

This will later e used to disqualify certain nodes from being takeover 
targets when we perform reallocation.

---

Summary of changes:
 include/ctdb_protocol.h |2 +
 server/ctdb_takeover.c  |   65 ++
 2 files changed, 50 insertions(+), 17 deletions(-)


Changeset truncated at 500 lines:

diff --git a/include/ctdb_protocol.h b/include/ctdb_protocol.h
index a153f01..ae989a9 100644
--- a/include/ctdb_protocol.h
+++ b/include/ctdb_protocol.h
@@ -559,6 +559,8 @@ struct ctdb_node_map {
 #define NODE_FLAGS_DISABLED
(NODE_FLAGS_UNHEALTHY|NODE_FLAGS_PERMANENTLY_DISABLED)
 #define NODE_FLAGS_INACTIVE
(NODE_FLAGS_DELETED|NODE_FLAGS_DISCONNECTED|NODE_FLAGS_BANNED|NODE_FLAGS_STOPPED)
 
+#define NODE_FLAGS_NOIPFAILBACK0x0100 /* this node can not 
be failed back onto, this flag is ONLY valid within the recovery daemon */
+
 
 struct ctdb_public_ip {
uint32_t pnn;
diff --git a/server/ctdb_takeover.c b/server/ctdb_takeover.c
index bff22a8..dba375b 100644
--- a/server/ctdb_takeover.c
+++ b/server/ctdb_takeover.c
@@ -1980,19 +1980,6 @@ try_again:
basic_allocate_unassigned(ctdb, nodemap, mask, all_ips);
}
 
-   /* If we dont want ips to fail back after a node becomes healthy
-  again, we wont even try to reallocat the ip addresses so that
-  they are evenly spread out.
-  This can NOT be used at the same time as DeterministicIPs !
-   */
-   if (1 == ctdb-tunable.no_ip_failback) {
-   if (1 == ctdb-tunable.deterministic_public_ips) {
-   DEBUG(DEBUG_ERR, (ERROR: You can not use 
'DeterministicIPs' and 'NoIPFailback' at the same time\n));
-   }
-   goto finished;
-   }
-
-
/* now, try to make sure the ip adresses are evenly distributed
   across the node.
*/
@@ -2008,16 +1995,37 @@ try_again:
 
/* finished distributing the public addresses, now just send the 
   info out to the nodes
-   */
-finished:
-
-   /* at this point -pnn is the node which will own each IP
+  at this point -pnn is the node which will own each IP
   or -1 if there is no node that can cover this ip
*/
 
return;
 }
 
+static void noipfailback_cb(struct ctdb_context *ctdb, uint32_t pnn, int32_t 
res, TDB_DATA outdata, void *callback)
+{
+   struct ctdb_node_map *nodemap = (struct ctdb_node_map *)callback;
+
+   if (res != 0) {
+   DEBUG(DEBUG_ERR,(Failure to read NoIPFailback tunable from 
remote node %d\n, pnn));
+   return;
+   }
+
+   if (outdata.dsize != sizeof(uint32_t)) {
+   DEBUG(DEBUG_ERR,(Wrong size of returned data when reading 
NoIPFailback tunable from node %d. Expected %d bytes but received %d bytes\n, 
pnn, (int)sizeof(uint32_t), (int)outdata.dsize));
+   return;
+   }
+
+   if (pnn = nodemap-num) {
+   DEBUG(DEBUG_ERR,(Got NoIPFailback reply from node %d but 
nodemap only has %d entries\n, pnn, nodemap-num));
+   return;
+   }
+
+   if (*(uint32_t *)outdata.dptr != 0) {
+   nodemap-nodes[pnn].flags |= NODE_FLAGS_NOIPFAILBACK;
+   }
+}
+
 /*
   make any IP alias changes for public addresses that are necessary 
  */
@@ -2026,6 +2034,7 @@ int ctdb_takeover_run(struct ctdb_context *ctdb, struct 
ctdb_node_map *nodemap)
int i;
struct ctdb_public_ip ip;
struct ctdb_public_ipv4 ipv4;
+   struct ctdb_control_get_tunable *t;
uint32_t *nodes;
struct ctdb_public_ip_list *all_ips, *tmp_ip;
TDB_DATA data;
@@ -2042,6 +2051,28 @@ int ctdb_takeover_run(struct ctdb_context *ctdb, struct 
ctdb_node_map *nodemap)
goto ipreallocated;
}
 
+
+   /* assume all nodes do support failback */
+   for (i=0;inodemap-num;i++) {
+   nodemap-nodes[i].flags = ~NODE_FLAGS_NOIPFAILBACK;
+   }
+   data.dsize = offsetof(struct ctdb_control_get_tunable, name) + 
strlen(NoIPFailback

[SCM] CTDB repository - branch 1.13 updated - ctdb-1.13-15-g77db0a8

2012-03-21 Thread Ronnie Sahlberg
The branch, 1.13 has been updated
   via  77db0a8e48b2693c50bcb59ad72ab5d603723135 (commit)
  from  ae86409ded3a242dd2f7452287ebdb55e006b055 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=1.13


- Log -
commit 77db0a8e48b2693c50bcb59ad72ab5d603723135
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Thu Mar 22 09:22:35 2012 +1100

NoIPFailback: Exclude nodes which have NoIPFailback as failback targets 
during reallocation

---

Summary of changes:
 server/ctdb_takeover.c |   22 ++
 1 files changed, 22 insertions(+), 0 deletions(-)


Changeset truncated at 500 lines:

diff --git a/server/ctdb_takeover.c b/server/ctdb_takeover.c
index dba375b..0ed5132 100644
--- a/server/ctdb_takeover.c
+++ b/server/ctdb_takeover.c
@@ -1175,6 +1175,12 @@ static int find_takeover_node(struct ctdb_context *ctdb,
 
pnn= -1;
for (i=0;inodemap-num;i++) {
+   if (nodemap-nodes[i].flags  NODE_FLAGS_NOIPFAILBACK) {
+   /* This node is not allowed to takeover any addresses
+   */
+   continue;
+   }
+
if (nodemap-nodes[i].flags  mask) {
/* This node is not healty and can not be used to serve
   a public address 
@@ -1465,6 +1471,11 @@ static bool basic_failback(struct ctdb_context *ctdb,
continue;
}
 
+   /* Only check nodes that are allowed to takeover an ip 
*/
+   if (nodemap-nodes[i].flags  NODE_FLAGS_NOIPFAILBACK) {
+   continue;
+   }
+
/* only check nodes that can actually serve this ip */
if (can_node_serve_ip(ctdb, i, tmp_ip)) {
/* no it couldnt   so skip to the next node */
@@ -1640,6 +1651,11 @@ static void lcp2_allocate_unassigned(struct ctdb_context 
*ctdb,
}
 
for (dstnode=0; dstnode  nodemap-num; dstnode++) {
+   /* Only check nodes that are allowed to 
takeover an ip */
+   if (nodemap-nodes[dstnode].flags  
NODE_FLAGS_NOIPFAILBACK) {
+   continue;
+   }
+
/* only check nodes that can actually serve 
this ip */
if (can_node_serve_ip(ctdb, dstnode, tmp_ip)) {
/* no it couldnt   so skip to the next 
node */
@@ -1748,6 +1764,12 @@ static bool lcp2_failback_candidate(struct ctdb_context 
*ctdb,
if (! newly_healthy[dstnode]) {
continue;
}
+
+   /* Only check nodes that are allowed to takeover an ip 
*/
+   if (nodemap-nodes[dstnode].flags  
NODE_FLAGS_NOIPFAILBACK) {
+   continue;
+   }
+
/* only check nodes that can actually serve this ip */
if (can_node_serve_ip(ctdb, dstnode, tmp_ip)) {
/* no it couldnt   so skip to the next node */


-- 
CTDB repository


[SCM] CTDB repository - branch master updated - ctdb-1.13-19-gc262c29

2012-03-21 Thread Ronnie Sahlberg
The branch, master has been updated
   via  c262c29773d1608e7ce04bdfb7f4469df0a9637b (commit)
  from  668f3e88a9e5f598706952b7140547640c85a5ed (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=master


- Log -
commit c262c29773d1608e7ce04bdfb7f4469df0a9637b
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Thu Mar 22 09:22:35 2012 +1100

NoIPFailback: Exclude nodes which have NoIPFailback as failback targets 
during reallocation

---

Summary of changes:
 server/ctdb_takeover.c |   22 ++
 1 files changed, 22 insertions(+), 0 deletions(-)


Changeset truncated at 500 lines:

diff --git a/server/ctdb_takeover.c b/server/ctdb_takeover.c
index dba375b..0ed5132 100644
--- a/server/ctdb_takeover.c
+++ b/server/ctdb_takeover.c
@@ -1175,6 +1175,12 @@ static int find_takeover_node(struct ctdb_context *ctdb,
 
pnn= -1;
for (i=0;inodemap-num;i++) {
+   if (nodemap-nodes[i].flags  NODE_FLAGS_NOIPFAILBACK) {
+   /* This node is not allowed to takeover any addresses
+   */
+   continue;
+   }
+
if (nodemap-nodes[i].flags  mask) {
/* This node is not healty and can not be used to serve
   a public address 
@@ -1465,6 +1471,11 @@ static bool basic_failback(struct ctdb_context *ctdb,
continue;
}
 
+   /* Only check nodes that are allowed to takeover an ip 
*/
+   if (nodemap-nodes[i].flags  NODE_FLAGS_NOIPFAILBACK) {
+   continue;
+   }
+
/* only check nodes that can actually serve this ip */
if (can_node_serve_ip(ctdb, i, tmp_ip)) {
/* no it couldnt   so skip to the next node */
@@ -1640,6 +1651,11 @@ static void lcp2_allocate_unassigned(struct ctdb_context 
*ctdb,
}
 
for (dstnode=0; dstnode  nodemap-num; dstnode++) {
+   /* Only check nodes that are allowed to 
takeover an ip */
+   if (nodemap-nodes[dstnode].flags  
NODE_FLAGS_NOIPFAILBACK) {
+   continue;
+   }
+
/* only check nodes that can actually serve 
this ip */
if (can_node_serve_ip(ctdb, dstnode, tmp_ip)) {
/* no it couldnt   so skip to the next 
node */
@@ -1748,6 +1764,12 @@ static bool lcp2_failback_candidate(struct ctdb_context 
*ctdb,
if (! newly_healthy[dstnode]) {
continue;
}
+
+   /* Only check nodes that are allowed to takeover an ip 
*/
+   if (nodemap-nodes[dstnode].flags  
NODE_FLAGS_NOIPFAILBACK) {
+   continue;
+   }
+
/* only check nodes that can actually serve this ip */
if (can_node_serve_ip(ctdb, dstnode, tmp_ip)) {
/* no it couldnt   so skip to the next node */


-- 
CTDB repository


[SCM] CTDB repository - branch 1.13 updated - ctdb-1.13-17-g1d69f89

2012-03-21 Thread Ronnie Sahlberg
The branch, 1.13 has been updated
   via  1d69f897ce525a9d8967a365718143a3a4dcf9a8 (commit)
   via  c1e421b955c1de060af9ae4d6489f3170198f727 (commit)
  from  77db0a8e48b2693c50bcb59ad72ab5d603723135 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=1.13


- Log -
commit 1d69f897ce525a9d8967a365718143a3a4dcf9a8
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Thu Mar 22 11:12:37 2012 +1100

DOC: regenerate manpages

commit c1e421b955c1de060af9ae4d6489f3170198f727
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Thu Mar 22 11:00:27 2012 +1100

NoIPTakeover: change the tunable name for the dont allow failing addresses 
over onto the node to NoIPTakeover

---

Summary of changes:
 doc/ctdbd.1 |   11 +++-
 doc/ctdbd.1.html|  146 ---
 doc/ctdbd.1.xml |   11 +++-
 include/ctdb_private.h  |1 +
 include/ctdb_protocol.h |2 +-
 server/ctdb_takeover.c  |   48 ++--
 server/ctdb_tunables.c  |3 +-
 7 files changed, 128 insertions(+), 94 deletions(-)


Changeset truncated at 500 lines:

diff --git a/doc/ctdbd.1 b/doc/ctdbd.1
index 87f0bfa..7e5443e 100644
--- a/doc/ctdbd.1
+++ b/doc/ctdbd.1
@@ -2,12 +2,12 @@
 .\ Title: ctdbd
 .\Author: [FIXME: author] [see http://docbook.sf.net/el/author]
 .\ Generator: DocBook XSL Stylesheets v1.75.2 http://docbook.sf.net/
-.\  Date: 03/20/2012
+.\  Date: 03/22/2012
 .\Manual: CTDB - clustered TDB database
 .\Source: ctdb
 .\  Language: English
 .\
-.TH CTDBD 1 03/20/2012 ctdb CTDB \- clustered TDB database
+.TH CTDBD 1 03/22/2012 ctdb CTDB \- clustered TDB database
 .\ -
 .\ * Define some portability stuff
 .\ -
@@ -475,9 +475,14 @@ When you enable this tunable, CTDB will no longer attempt 
to rebalance the clust
 .PP
 Default: 0
 .PP
-When enabled, ctdb weill not perform failover or failback\. Even if a node 
fails while holding public IPs, ctdb will not recover the IPs or assign them to 
another node\.
+When enabled, ctdb will not perform failover or failback\. Even if a node 
fails while holding public IPs, ctdb will not recover the IPs or assign them to 
another node\.
 .PP
 When you enable this tunable, CTDB will no longer attempt to recover the 
cluster by failing IP addresses over to other nodes\. This leads to a service 
outage until the administrator has manually performed failover to replacement 
nodes using the \*(Aqctdb moveip\*(Aq command\.
+.SS NoIPTakeover
+.PP
+Default: 0
+.PP
+When set to 1, ctdb will allow ip addresses to be failed over onto this 
node\. Any ip addresses that the node currently hosts will remain on the node 
but no new ip addresses can be failed over onto the node\.
 .SS VerboseMemoryNames
 .PP
 Default: 0
diff --git a/doc/ctdbd.1.html b/doc/ctdbd.1.html
index 11f3693..add708b 100644
--- a/doc/ctdbd.1.html
+++ b/doc/ctdbd.1.html
@@ -1,4 +1,4 @@
-htmlheadmeta http-equiv=Content-Type content=text/html; 
charset=ISO-8859-1titlectdbd/titlemeta name=generator content=DocBook 
XSL Stylesheets V1.75.2/headbody bgcolor=white text=black 
link=#FF vlink=#840084 alink=#FFdiv class=refentry 
title=ctdbda name=ctdbd.1/adiv class=titlepage/divdiv 
class=refnamedivh2Name/h2pctdbd #8212; The CTDB cluster 
daemon/p/divdiv class=refsynopsisdiv 
title=Synopsish2Synopsis/h2div class=cmdsynopsispcode 
class=commandctdbd/code /p/divdiv class=cmdsynopsispcode 
class=commandctdbd/code  [-? --help] [-d --debug=lt;INTEGERgt;] 
{--dbdir=lt;directorygt;} {--dbdir-persistent=lt;directorygt;} 
[--event-script-dir=lt;directorygt;] [-i --interactive] 
[--listen=lt;addressgt;] [--logfile=lt;filenamegt;] [--lvs] 
{--nlist=lt;filenamegt;} [--no-lmaster] [--no-recmaster] [--nosetsched] 
{--notification-script=lt;filenamegt;} [--public-add
 resses=lt;filenamegt;] [--public-interface=lt;interfacegt;] 
{--reclock=lt;filenamegt;} [--single-public-ip=lt;addressgt;] 
[--socket=lt;filenamegt;] [--start-as-disabled] [--start-as-stopped] 
[--syslog] [--log-ringbuf-size=lt;num-entriesgt;] [--torture] 
[--transport=lt;STRINGgt;] [--usage]/p/div/divdiv class=refsect1 
title=DESCRIPTIONa name=id386691/ah2DESCRIPTION/h2p
+htmlheadmeta http-equiv=Content-Type content=text/html; 
charset=ISO-8859-1titlectdbd/titlemeta name=generator content=DocBook 
XSL Stylesheets V1.75.2/headbody bgcolor=white text=black 
link=#FF vlink=#840084 alink=#FFdiv class=refentry 
title=ctdbda name=ctdbd.1/adiv class=titlepage/divdiv 
class=refnamedivh2Name/h2pctdbd #8212; The CTDB cluster 
daemon/p/divdiv class=refsynopsisdiv 
title=Synopsish2Synopsis/h2div class=cmdsynopsispcode 
class=commandctdbd/code /p/divdiv class=cmdsynopsispcode 
class=commandctdbd/code  [-? --help] [-d --debug=lt

[SCM] CTDB repository - branch master updated - ctdb-1.13-21-g270edc8

2012-03-21 Thread Ronnie Sahlberg
The branch, master has been updated
   via  270edc87e6bcfd6e76c526bf86df865e4d47d11c (commit)
   via  35592e618cfd827b6978af6332f80504f232c46a (commit)
  from  c262c29773d1608e7ce04bdfb7f4469df0a9637b (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=master


- Log -
commit 270edc87e6bcfd6e76c526bf86df865e4d47d11c
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Thu Mar 22 11:12:37 2012 +1100

DOC: regenerate manpages

commit 35592e618cfd827b6978af6332f80504f232c46a
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Thu Mar 22 11:00:27 2012 +1100

NoIPTakeover: change the tunable name for the dont allow failing addresses 
over onto the node to NoIPTakeover

---

Summary of changes:
 doc/ctdbd.1 |   11 +++-
 doc/ctdbd.1.html|  146 ---
 doc/ctdbd.1.xml |   11 +++-
 include/ctdb_private.h  |1 +
 include/ctdb_protocol.h |2 +-
 server/ctdb_takeover.c  |   48 ++--
 server/ctdb_tunables.c  |3 +-
 7 files changed, 128 insertions(+), 94 deletions(-)


Changeset truncated at 500 lines:

diff --git a/doc/ctdbd.1 b/doc/ctdbd.1
index 87f0bfa..7e5443e 100644
--- a/doc/ctdbd.1
+++ b/doc/ctdbd.1
@@ -2,12 +2,12 @@
 .\ Title: ctdbd
 .\Author: [FIXME: author] [see http://docbook.sf.net/el/author]
 .\ Generator: DocBook XSL Stylesheets v1.75.2 http://docbook.sf.net/
-.\  Date: 03/20/2012
+.\  Date: 03/22/2012
 .\Manual: CTDB - clustered TDB database
 .\Source: ctdb
 .\  Language: English
 .\
-.TH CTDBD 1 03/20/2012 ctdb CTDB \- clustered TDB database
+.TH CTDBD 1 03/22/2012 ctdb CTDB \- clustered TDB database
 .\ -
 .\ * Define some portability stuff
 .\ -
@@ -475,9 +475,14 @@ When you enable this tunable, CTDB will no longer attempt 
to rebalance the clust
 .PP
 Default: 0
 .PP
-When enabled, ctdb weill not perform failover or failback\. Even if a node 
fails while holding public IPs, ctdb will not recover the IPs or assign them to 
another node\.
+When enabled, ctdb will not perform failover or failback\. Even if a node 
fails while holding public IPs, ctdb will not recover the IPs or assign them to 
another node\.
 .PP
 When you enable this tunable, CTDB will no longer attempt to recover the 
cluster by failing IP addresses over to other nodes\. This leads to a service 
outage until the administrator has manually performed failover to replacement 
nodes using the \*(Aqctdb moveip\*(Aq command\.
+.SS NoIPTakeover
+.PP
+Default: 0
+.PP
+When set to 1, ctdb will allow ip addresses to be failed over onto this 
node\. Any ip addresses that the node currently hosts will remain on the node 
but no new ip addresses can be failed over onto the node\.
 .SS VerboseMemoryNames
 .PP
 Default: 0
diff --git a/doc/ctdbd.1.html b/doc/ctdbd.1.html
index 11f3693..add708b 100644
--- a/doc/ctdbd.1.html
+++ b/doc/ctdbd.1.html
@@ -1,4 +1,4 @@
-htmlheadmeta http-equiv=Content-Type content=text/html; 
charset=ISO-8859-1titlectdbd/titlemeta name=generator content=DocBook 
XSL Stylesheets V1.75.2/headbody bgcolor=white text=black 
link=#FF vlink=#840084 alink=#FFdiv class=refentry 
title=ctdbda name=ctdbd.1/adiv class=titlepage/divdiv 
class=refnamedivh2Name/h2pctdbd #8212; The CTDB cluster 
daemon/p/divdiv class=refsynopsisdiv 
title=Synopsish2Synopsis/h2div class=cmdsynopsispcode 
class=commandctdbd/code /p/divdiv class=cmdsynopsispcode 
class=commandctdbd/code  [-? --help] [-d --debug=lt;INTEGERgt;] 
{--dbdir=lt;directorygt;} {--dbdir-persistent=lt;directorygt;} 
[--event-script-dir=lt;directorygt;] [-i --interactive] 
[--listen=lt;addressgt;] [--logfile=lt;filenamegt;] [--lvs] 
{--nlist=lt;filenamegt;} [--no-lmaster] [--no-recmaster] [--nosetsched] 
{--notification-script=lt;filenamegt;} [--public-add
 resses=lt;filenamegt;] [--public-interface=lt;interfacegt;] 
{--reclock=lt;filenamegt;} [--single-public-ip=lt;addressgt;] 
[--socket=lt;filenamegt;] [--start-as-disabled] [--start-as-stopped] 
[--syslog] [--log-ringbuf-size=lt;num-entriesgt;] [--torture] 
[--transport=lt;STRINGgt;] [--usage]/p/div/divdiv class=refsect1 
title=DESCRIPTIONa name=id386691/ah2DESCRIPTION/h2p
+htmlheadmeta http-equiv=Content-Type content=text/html; 
charset=ISO-8859-1titlectdbd/titlemeta name=generator content=DocBook 
XSL Stylesheets V1.75.2/headbody bgcolor=white text=black 
link=#FF vlink=#840084 alink=#FFdiv class=refentry 
title=ctdbda name=ctdbd.1/adiv class=titlepage/divdiv 
class=refnamedivh2Name/h2pctdbd #8212; The CTDB cluster 
daemon/p/divdiv class=refsynopsisdiv 
title=Synopsish2Synopsis/h2div class=cmdsynopsispcode 
class=commandctdbd/code /p/divdiv class=cmdsynopsispcode 
class=commandctdbd/code  [-? --help] [-d --debug=lt

[SCM] CTDB repository - branch 1.13 updated - ctdb-1.13-66-g41ee3cb

2012-03-21 Thread Ronnie Sahlberg
The branch, 1.13 has been updated
   via  41ee3cb65b63605ab968cf1eef05a75256d4db6c (commit)
   via  f6d4cf009d63605f8c1246c59c53b1f82989d2a1 (commit)
   via  7d7488a49bc7e799f39f3da3fa81be7a2e14480a (commit)
   via  da2e9650a97f99e7d694659926d9958927edd8ad (commit)
   via  0142729c96b8bfecfabe35439071557bcc6ed4da (commit)
   via  1db9a533aacc6e02ddbadf76241429144a949e1c (commit)
   via  299ce89fb3deeefcc7c01b20197dc6adee1e8194 (commit)
   via  79eec235ac9b051333751186c32924d92e6ebeff (commit)
   via  0fc20ab5207d4f812bf62b90a66c707f76742f84 (commit)
   via  9c30bf7fd3d94884e27a258571a28dca5e460177 (commit)
   via  6d4b4b1293ba960d8fdf81958d6e8dd23c9d771a (commit)
   via  38396185effbcf313678c2e94d4bd20f534c4f03 (commit)
   via  d132a8f4a8c0573ac447713a71911f02b6f0d73c (commit)
   via  236d572556e447e2d82d14d2c8a40ed8cb98944d (commit)
   via  351ca413eec460330571ca8b01ad269728fe15df (commit)
   via  d4621277240721e6d130a930b0100506b64467ea (commit)
   via  bdb4cdaf2aed79c8de6a8db8c01685b242808310 (commit)
   via  d034845ecea66b47004bc73f2554914a397b1c9d (commit)
   via  ac2cd2a8cbed97348ceae54167ad83cb074ff6b3 (commit)
   via  26292f362890ca6cb65d0028d50fa97458e0449f (commit)
   via  8b8e17bea87a8c16f7c0f54fcfe29190f203f673 (commit)
   via  f4949b3c1b36949544e98c9fbb64447f94f9e432 (commit)
   via  994492f79275fe84155d842f6bc288c1858217dd (commit)
   via  93f90caf91246074d9359bf31a39b2621242 (commit)
   via  2a2ea6c61a05af2d0765e964abcc7ef04047431e (commit)
   via  45c3476d12bf0f52966b72d286f101fce1382cd2 (commit)
   via  0f7cbbb55f26cf3c953e98fe5e7eaa12f59fbf78 (commit)
   via  97b0c138cb97e30db27c40b4ee1481109ae90c78 (commit)
   via  270edc87e6bcfd6e76c526bf86df865e4d47d11c (commit)
   via  35592e618cfd827b6978af6332f80504f232c46a (commit)
   via  c262c29773d1608e7ce04bdfb7f4469df0a9637b (commit)
   via  668f3e88a9e5f598706952b7140547640c85a5ed (commit)
   via  0ae2a46e37df8d84ce0c76b7a5d32d6e93af709b (commit)
   via  e6a8306257dece02313043274789b52cef1a5647 (commit)
   via  d0d98f7e45e5084b81335b004d50bddc80cdc219 (commit)
   via  cf96d6380de974c1abff7a33c62a7e31ac980551 (commit)
   via  035c0d981bde8c0eee8b3f24ba8e2dc817e5b504 (commit)
   via  1c976d83b1d7dac6f0ef81306774998e4c8b56a1 (commit)
   via  05923a14bc8ae8e3c5cda28ff4e0fb0d473d5f7d (commit)
   via  c37aa6f3738693653f64c2fa015ace061da38b5a (commit)
   via  eafd7bbaaa5931546a96c8beae3cf9a39a49c925 (commit)
   via  7d3931298e6477d92f43652c3006b0c426cb1307 (commit)
   via  af72f8bc05f78983f9624d81788f9d89af474b11 (commit)
   via  f06634951331232cddf0b48eac3552b92aca5b93 (commit)
   via  68390cb1e8c7f30be034b8ca6c9215b40147305b (commit)
   via  fb00e1290fcea3386132a46c883994019a43799a (commit)
   via  a7ad2fb75f06791508dd928d2a0c305fc7f7b814 (commit)
   via  db69a39ca010b2c8853b13466e30792dda350cdb (commit)
   via  a0340a50c2acd9ccc281faef032a364254f7f95a (commit)
  from  1d69f897ce525a9d8967a365718143a3a4dcf9a8 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=1.13


- Log -
commit 41ee3cb65b63605ab968cf1eef05a75256d4db6c
Merge: f6d4cf009d63605f8c1246c59c53b1f82989d2a1 
7d7488a49bc7e799f39f3da3fa81be7a2e14480a
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Thu Mar 22 15:40:55 2012 +1100

Merge remote branch 'martins/policy_routing' into 1.13

commit f6d4cf009d63605f8c1246c59c53b1f82989d2a1
Author: Martin Schwenke mar...@meltin.net
Date:   Thu Mar 22 15:27:25 2012 +1100

Undo damage done by d8d37493478a26c5f1809a5f3df89ffd6e149281

The implementation of DisableIPFailover got intermingled with
--nopublicipcheck.  This just looks wrong - Ronnie must have been
having a bad day.  :-)

Signed-off-by: Martin Schwenke mar...@meltin.net

---

Summary of changes:
 .gitignore |2 +
 Makefile.in|6 +-
 config/README  |6 -
 config/events.d/13.per_ip_routing  |  662 
 config/functions   |  197 +++
 config/interface_modify.sh |  142 -
 include/ctdb_private.h |1 +
 packaging/RPM/ctdb.spec.in |1 -
 server/ctdb_recoverd.c |2 +-
 server/ctdbd.c |4 +-
 tests/eventscripts/common.sh   |  188 --
 tests/eventscripts/etc-ctdb/public_addresses   |5 +
 .../multievent/13.per_ip_routing.001.sh|   16 +
 .../multievent/13.per_ip_routing.002.sh|   14 +
 .../multievent/13.per_ip_routing.003.sh

[SCM] CTDB repository - branch master updated - ctdb-1.13-49-g7293762

2012-03-21 Thread Ronnie Sahlberg
The branch, master has been updated
   via  7293762046e25a9c96e6417e34e18c214ebc6e55 (commit)
   via  5083b266dd68b292c4275505f3d1b878dbf12f11 (commit)
   via  7d7488a49bc7e799f39f3da3fa81be7a2e14480a (commit)
   via  da2e9650a97f99e7d694659926d9958927edd8ad (commit)
   via  0142729c96b8bfecfabe35439071557bcc6ed4da (commit)
   via  1db9a533aacc6e02ddbadf76241429144a949e1c (commit)
   via  299ce89fb3deeefcc7c01b20197dc6adee1e8194 (commit)
   via  79eec235ac9b051333751186c32924d92e6ebeff (commit)
   via  0fc20ab5207d4f812bf62b90a66c707f76742f84 (commit)
   via  9c30bf7fd3d94884e27a258571a28dca5e460177 (commit)
   via  6d4b4b1293ba960d8fdf81958d6e8dd23c9d771a (commit)
   via  38396185effbcf313678c2e94d4bd20f534c4f03 (commit)
   via  d132a8f4a8c0573ac447713a71911f02b6f0d73c (commit)
   via  236d572556e447e2d82d14d2c8a40ed8cb98944d (commit)
   via  351ca413eec460330571ca8b01ad269728fe15df (commit)
   via  d4621277240721e6d130a930b0100506b64467ea (commit)
   via  bdb4cdaf2aed79c8de6a8db8c01685b242808310 (commit)
   via  d034845ecea66b47004bc73f2554914a397b1c9d (commit)
   via  ac2cd2a8cbed97348ceae54167ad83cb074ff6b3 (commit)
   via  26292f362890ca6cb65d0028d50fa97458e0449f (commit)
   via  8b8e17bea87a8c16f7c0f54fcfe29190f203f673 (commit)
   via  f4949b3c1b36949544e98c9fbb64447f94f9e432 (commit)
   via  994492f79275fe84155d842f6bc288c1858217dd (commit)
   via  93f90caf91246074d9359bf31a39b2621242 (commit)
   via  2a2ea6c61a05af2d0765e964abcc7ef04047431e (commit)
   via  45c3476d12bf0f52966b72d286f101fce1382cd2 (commit)
   via  0f7cbbb55f26cf3c953e98fe5e7eaa12f59fbf78 (commit)
   via  97b0c138cb97e30db27c40b4ee1481109ae90c78 (commit)
  from  270edc87e6bcfd6e76c526bf86df865e4d47d11c (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=master


- Log -
commit 7293762046e25a9c96e6417e34e18c214ebc6e55
Merge: 5083b266dd68b292c4275505f3d1b878dbf12f11 
7d7488a49bc7e799f39f3da3fa81be7a2e14480a
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Thu Mar 22 15:39:59 2012 +1100

Merge remote branch 'martins/policy_routing'

commit 5083b266dd68b292c4275505f3d1b878dbf12f11
Author: Martin Schwenke mar...@meltin.net
Date:   Thu Mar 22 15:27:25 2012 +1100

Undo damage done by d8d37493478a26c5f1809a5f3df89ffd6e149281

The implementation of DisableIPFailover got intermingled with
--nopublicipcheck.  This just looks wrong - Ronnie must have been
having a bad day.  :-)

Signed-off-by: Martin Schwenke mar...@meltin.net

---

Summary of changes:
 .gitignore |2 +
 Makefile.in|1 -
 config/README  |6 -
 config/events.d/13.per_ip_routing  |  662 
 config/functions   |  197 +++
 config/interface_modify.sh |  142 -
 include/ctdb_private.h |1 +
 packaging/RPM/ctdb.spec.in |1 -
 server/ctdb_recoverd.c |2 +-
 server/ctdbd.c |4 +-
 tests/eventscripts/common.sh   |  188 --
 tests/eventscripts/etc-ctdb/public_addresses   |5 +
 .../multievent/13.per_ip_routing.001.sh|   16 +
 .../multievent/13.per_ip_routing.002.sh|   14 +
 .../multievent/13.per_ip_routing.003.sh|   24 +
 .../multievent/13.per_ip_routing.004.sh|   25 +
 .../multievent/13.per_ip_routing.005.sh|   38 ++
 .../multievent/13.per_ip_routing.006.sh|   38 ++
 .../multievent/13.per_ip_routing.007.sh|   40 ++
 .../multievent/13.per_ip_routing.008.sh|   42 ++
 .../multievent/13.per_ip_routing.009.sh|   44 ++
 .../multievent/13.per_ip_routing.010.sh|   52 ++
 .../multievent/13.per_ip_routing.011.sh|   41 ++
 .../multievent/13.per_ip_routing.012.sh|   48 ++
 tests/eventscripts/run_tests.sh|   13 +
 .../simple/10.interface.monitor.015.sh |2 +-
 .../eventscripts/simple/10.interface.takeip.003.sh |6 +-
 tests/eventscripts/stubs/ctdb  |  208 +--
 tests/eventscripts/stubs/ip|  598 +-
 29 files changed, 1512 insertions(+), 948 deletions(-)
 delete mode 100755 config/interface_modify.sh
 create mode 100755 tests/eventscripts/multievent/13.per_ip_routing.001.sh
 create mode 100755 tests/eventscripts/multievent/13.per_ip_routing.002.sh
 create mode 100755 tests/eventscripts/multievent/13.per_ip_routing.003.sh
 create mode 100755 tests/eventscripts/multievent/13.per_ip_routing.004.sh

[SCM] CTDB repository - branch master updated - ctdb-1.13-17-g0ae2a46

2012-03-20 Thread Ronnie Sahlberg
The branch, master has been updated
   via  0ae2a46e37df8d84ce0c76b7a5d32d6e93af709b (commit)
   via  e6a8306257dece02313043274789b52cef1a5647 (commit)
   via  d0d98f7e45e5084b81335b004d50bddc80cdc219 (commit)
   via  cf96d6380de974c1abff7a33c62a7e31ac980551 (commit)
   via  035c0d981bde8c0eee8b3f24ba8e2dc817e5b504 (commit)
   via  1c976d83b1d7dac6f0ef81306774998e4c8b56a1 (commit)
   via  05923a14bc8ae8e3c5cda28ff4e0fb0d473d5f7d (commit)
   via  c37aa6f3738693653f64c2fa015ace061da38b5a (commit)
   via  eafd7bbaaa5931546a96c8beae3cf9a39a49c925 (commit)
   via  7d3931298e6477d92f43652c3006b0c426cb1307 (commit)
  from  af72f8bc05f78983f9624d81788f9d89af474b11 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=master


- Log -
commit 0ae2a46e37df8d84ce0c76b7a5d32d6e93af709b
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Tue Mar 20 17:43:26 2012 +1100

DOC  regenerate the manpage

commit e6a8306257dece02313043274789b52cef1a5647
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Tue Mar 20 17:07:13 2012 +1100

STICKY   document the sticky record parameters

commit d0d98f7e45e5084b81335b004d50bddc80cdc219
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Tue Mar 20 16:58:35 2012 +1100

STICKY: add prototype code to make records stick to a node to calm down 
if they are found to be very hot and accessed by a lot of clients.

This can improve performance and stop clients from having to chase a 
rapidly migrating/bouncing record

commit cf96d6380de974c1abff7a33c62a7e31ac980551
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Tue Mar 20 12:25:47 2012 +1100

DOC: Document the MaxLACount tunable variable

commit 035c0d981bde8c0eee8b3f24ba8e2dc817e5b504
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Tue Mar 20 12:20:55 2012 +1100

LACOUNT:  Add back lacount mechanism to defer migrating a fetched/read copy 
until after default of 20 consecutive requests from the same node

This can improve performance slightly on certain workloads where smbds 
frequently read from the same record

commit 1c976d83b1d7dac6f0ef81306774998e4c8b56a1
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Tue Mar 20 12:08:12 2012 +1100

STATISTICS: add per-db hop count statistics

commit 05923a14bc8ae8e3c5cda28ff4e0fb0d473d5f7d
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Tue Mar 20 11:43:09 2012 +1100

CTDB: change how the hop count buckets are displayed in ctdb statistics

commit c37aa6f3738693653f64c2fa015ace061da38b5a
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Tue Mar 20 11:38:20 2012 +1100

DOC document the FetchCollapse tunable

commit eafd7bbaaa5931546a96c8beae3cf9a39a49c925
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Tue Mar 20 11:31:59 2012 +1100

FETCH COLLAPSE : Change the fetch-lock collapse to collapse ALL fetches, 
including fetch-locks into a single command in flight per record. Also add a 
tunable to enable/disable this optimization for hot records

commit 7d3931298e6477d92f43652c3006b0c426cb1307
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Wed Mar 7 17:02:41 2012 +1100

add max hop count buckets to see how bad hopcounts are

---

Summary of changes:
 client/ctdb_client.c  |   52 -
 common/ctdb_ltdb.c|1 +
 doc/ctdbd.1   |  684 -
 doc/ctdbd.1.html  |  175 +++-
 doc/ctdbd.1.xml   |   71 +
 include/ctdb_client.h |6 +
 include/ctdb_private.h|   11 +-
 include/ctdb_protocol.h   |8 +-
 server/ctdb_call.c|  282 ++-
 server/ctdb_control.c |   11 +
 server/ctdb_daemon.c  |   13 +-
 server/ctdb_ltdb_server.c |   22 ++
 server/ctdb_recover.c |3 +
 server/ctdb_tunables.c|7 +-
 tools/ctdb.c  |   85 ++-
 15 files changed, 1083 insertions(+), 348 deletions(-)


Changeset truncated at 500 lines:

diff --git a/client/ctdb_client.c b/client/ctdb_client.c
index 1b41439..b01e258 100644
--- a/client/ctdb_client.c
+++ b/client/ctdb_client.c
@@ -71,7 +71,7 @@ struct ctdb_req_header *_ctdbd_allocate_pkt(struct 
ctdb_context *ctdb,
 */
 int ctdb_call_local(struct ctdb_db_context *ctdb_db, struct ctdb_call *call,
struct ctdb_ltdb_header *header, TALLOC_CTX *mem_ctx,
-   TDB_DATA *data, bool updatetdb)
+   TDB_DATA *data, bool updatetdb, uint32_t caller)
 {
struct ctdb_call_info *c;
struct ctdb_registered_call *fn;
@@ -106,7 +106,15 @@ int ctdb_call_local(struct ctdb_db_context *ctdb_db, 
struct ctdb_call *call,
}
 
/* we need to force the record to be written out if this was a remote 
access */
-   if (c-new_data == NULL) {
+   if (header-laccessor

[SCM] CTDB repository - branch 1.13 updated - ctdb-1.13-12-g6802b43

2012-03-20 Thread Ronnie Sahlberg
The branch, 1.13 has been updated
   via  6802b43c09cd11cc3074882e7a43fae52dc914a5 (commit)
   via  3fa9a3654a2a647ed326dcc17014abdcf50902ac (commit)
   via  fea93c966d21e0b51790a92a828057b075effc56 (commit)
   via  6f857bfc9f6bc7bc45891439463757d61848850a (commit)
   via  b2cc874c5850ba7c7e893eab68a864858f47ad5b (commit)
   via  ae0d8cf3b829ae33cc0c9c9e7277c99244fe9919 (commit)
   via  55d3038f58b94ecf420e6e605cc0432395cd9b3b (commit)
   via  3732e3a7f1267fb26ab58796d8b90f001399ba85 (commit)
   via  57ecdcc4edad4b56d727acd06d85a1473f2453bd (commit)
   via  0fd0135065acb55e54f4c25bcac7c563427c8008 (commit)
  from  8d606e5f71072ccc60bcac54dfd9e92bb699c6e5 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=1.13


- Log -
commit 6802b43c09cd11cc3074882e7a43fae52dc914a5
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Tue Mar 20 17:43:26 2012 +1100

DOC  regenerate the manpage

commit 3fa9a3654a2a647ed326dcc17014abdcf50902ac
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Tue Mar 20 17:07:13 2012 +1100

STICKY   document the sticky record parameters

commit fea93c966d21e0b51790a92a828057b075effc56
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Tue Mar 20 16:58:35 2012 +1100

STICKY: add prototype code to make records stick to a node to calm down 
if they are found to be very hot and accessed by a lot of clients.

This can improve performance and stop clients from having to chase a 
rapidly migrating/bouncing record

commit 6f857bfc9f6bc7bc45891439463757d61848850a
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Tue Mar 20 12:25:47 2012 +1100

DOC: Document the MaxLACount tunable variable

commit b2cc874c5850ba7c7e893eab68a864858f47ad5b
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Tue Mar 20 12:20:55 2012 +1100

LACOUNT:  Add back lacount mechanism to defer migrating a fetched/read copy 
until after default of 20 consecutive requests from the same node

This can improve performance slightly on certain workloads where smbds 
frequently read from the same record

commit ae0d8cf3b829ae33cc0c9c9e7277c99244fe9919
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Tue Mar 20 12:08:12 2012 +1100

STATISTICS: add per-db hop count statistics

commit 55d3038f58b94ecf420e6e605cc0432395cd9b3b
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Tue Mar 20 11:43:09 2012 +1100

CTDB: change how the hop count buckets are displayed in ctdb statistics

commit 3732e3a7f1267fb26ab58796d8b90f001399ba85
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Tue Mar 20 11:38:20 2012 +1100

DOC document the FetchCollapse tunable

commit 57ecdcc4edad4b56d727acd06d85a1473f2453bd
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Tue Mar 20 11:31:59 2012 +1100

FETCH COLLAPSE : Change the fetch-lock collapse to collapse ALL fetches, 
including fetch-locks into a single command in flight per record. Also add a 
tunable to enable/disable this optimization for hot records

commit 0fd0135065acb55e54f4c25bcac7c563427c8008
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Wed Mar 7 17:02:41 2012 +1100

add max hop count buckets to see how bad hopcounts are

---

Summary of changes:
 client/ctdb_client.c  |   52 -
 common/ctdb_ltdb.c|1 +
 doc/ctdbd.1   |  684 -
 doc/ctdbd.1.html  |  175 +++-
 doc/ctdbd.1.xml   |   71 +
 include/ctdb_client.h |6 +
 include/ctdb_private.h|   11 +-
 include/ctdb_protocol.h   |8 +-
 server/ctdb_call.c|  282 ++-
 server/ctdb_control.c |   11 +
 server/ctdb_daemon.c  |   13 +-
 server/ctdb_ltdb_server.c |   22 ++
 server/ctdb_recover.c |3 +
 server/ctdb_tunables.c|7 +-
 tools/ctdb.c  |   85 ++-
 15 files changed, 1083 insertions(+), 348 deletions(-)


Changeset truncated at 500 lines:

diff --git a/client/ctdb_client.c b/client/ctdb_client.c
index 1b41439..b01e258 100644
--- a/client/ctdb_client.c
+++ b/client/ctdb_client.c
@@ -71,7 +71,7 @@ struct ctdb_req_header *_ctdbd_allocate_pkt(struct 
ctdb_context *ctdb,
 */
 int ctdb_call_local(struct ctdb_db_context *ctdb_db, struct ctdb_call *call,
struct ctdb_ltdb_header *header, TALLOC_CTX *mem_ctx,
-   TDB_DATA *data, bool updatetdb)
+   TDB_DATA *data, bool updatetdb, uint32_t caller)
 {
struct ctdb_call_info *c;
struct ctdb_registered_call *fn;
@@ -106,7 +106,15 @@ int ctdb_call_local(struct ctdb_db_context *ctdb_db, 
struct ctdb_call *call,
}
 
/* we need to force the record to be written out if this was a remote 
access */
-   if (c-new_data == NULL) {
+   if (header-laccessor != caller

[SCM] CTDB repository - branch 1.2.39-28 updated - ctdb-1.9.1-505-g6b2c55d

2012-03-14 Thread Ronnie Sahlberg
The branch, 1.2.39-28 has been updated
   via  6b2c55d6ac396835597613b8f7fa271ac2bd6f4b (commit)
   via  cd8f268da8841d3803e64379ae8fb5384a7e7946 (commit)
   via  ecb58d0f17680e79f434a9d8f3048c71d967241b (commit)
   via  800dd35b7378af76fefb227b10de2f5ede82542c (commit)
   via  d156dec63857b19d1c55fbe089ef9f02e2b8bb50 (commit)
   via  7d9c1087ac7989bc705a373c4b4414f4f669f7c4 (commit)
   via  81d680873cb087e7baeb375eda1ebfc339ed10f4 (commit)
   via  0a460fc762ffab9b77431eeb48302bdac4682a10 (commit)
  from  635ff897405735aaf856f5dc900c2ae6aacf9243 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=1.2.39-28


- Log -
commit 6b2c55d6ac396835597613b8f7fa271ac2bd6f4b
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Thu Mar 15 10:22:16 2012 +1100

new version 1.2.39-30

commit cd8f268da8841d3803e64379ae8fb5384a7e7946
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Thu Mar 15 10:16:31 2012 +1100

Revert Add CTDB_CONTROL_CHECK_SRVID

This reverts commit 39d9e8b813e5957bd8319bd5ad6c4c9978c27dad.

commit ecb58d0f17680e79f434a9d8f3048c71d967241b
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Thu Mar 15 10:16:16 2012 +1100

Revert Record Fetch Collapse: Collapse multiple fetch request into one 
single request

This reverts commit 8bc2cca22faf33dad8908567fa30b55173cb3426.

commit 800dd35b7378af76fefb227b10de2f5ede82542c
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Thu Mar 15 10:15:56 2012 +1100

Revert Add back la-count based migration

This reverts commit 0c4ae347a20a780fe0326036de699e9f6cbb6d26.

commit d156dec63857b19d1c55fbe089ef9f02e2b8bb50
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Thu Mar 15 10:15:33 2012 +1100

Revert set max lacount to default to 20

This reverts commit 9e702ae900809bde8c3ba7e7a29a1171d0b7ed86.

commit 7d9c1087ac7989bc705a373c4b4414f4f669f7c4
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Thu Mar 15 10:14:44 2012 +1100

Revert Disable FetchLockCollapse by default and disable LACount too

This reverts commit 0a460fc762ffab9b77431eeb48302bdac4682a10.

commit 81d680873cb087e7baeb375eda1ebfc339ed10f4
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Tue Mar 13 17:33:20 2012 +1100

version 1.2.39-29   disable fetchlockcollapse and lacout by default

commit 0a460fc762ffab9b77431eeb48302bdac4682a10
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Tue Mar 13 17:32:41 2012 +1100

Disable FetchLockCollapse by default and disable LACount too

---

Summary of changes:
 client/ctdb_client.c   |   14 +--
 common/ctdb_ltdb.c |1 -
 include/ctdb_private.h |   10 +--
 include/ctdb_protocol.h|4 +-
 packaging/RPM/ctdb.spec.in |4 +-
 server/ctdb_call.c |   20 ++--
 server/ctdb_control.c  |3 -
 server/ctdb_daemon.c   |  239 
 server/ctdb_ltdb_server.c  |   11 --
 server/ctdb_tunables.c |4 +-
 10 files changed, 17 insertions(+), 293 deletions(-)


Changeset truncated at 500 lines:

diff --git a/client/ctdb_client.c b/client/ctdb_client.c
index 7103ef6..5e06604 100644
--- a/client/ctdb_client.c
+++ b/client/ctdb_client.c
@@ -72,7 +72,7 @@ struct ctdb_req_header *_ctdbd_allocate_pkt(struct 
ctdb_context *ctdb,
 */
 int ctdb_call_local(struct ctdb_db_context *ctdb_db, struct ctdb_call *call,
struct ctdb_ltdb_header *header, TALLOC_CTX *mem_ctx,
-   TDB_DATA *data, uint32_t caller)
+   TDB_DATA *data)
 {
struct ctdb_call_info *c;
struct ctdb_registered_call *fn;
@@ -106,15 +106,7 @@ int ctdb_call_local(struct ctdb_db_context *ctdb_db, 
struct ctdb_call *call,
}
 
/* we need to force the record to be written out if this was a remote 
access */
-   if (header-laccessor != caller) {
-   header-lacount = 0;
-   }
-   header-laccessor = caller;
-   header-lacount++;
-
-   /* we need to force the record to be written out if this was a remote 
access,
-  so that the lacount is updated */
-   if (c-new_data == NULL  header-laccessor != ctdb-pnn) {
+   if (c-new_data == NULL) {
c-new_data = c-record_data;
}
 
@@ -369,7 +361,7 @@ static struct ctdb_client_call_state 
*ctdb_client_call_local_send(struct ctdb_db
*(state-call) = *call;
state-ctdb_db = ctdb_db;
 
-   ret = ctdb_call_local(ctdb_db, state-call, header, state, data, 
ctdb-pnn);
+   ret = ctdb_call_local(ctdb_db, state-call, header, state, data);
 
return state;
 }
diff --git a/common/ctdb_ltdb.c b/common/ctdb_ltdb.c
index b0ab3e6..3ee7fe8 100644
--- a/common/ctdb_ltdb.c
+++ b/common/ctdb_ltdb.c
@@ -66,7 +66,6 @@ static void ltdb_initial_header(struct

[SCM] CTDB repository - branch master updated - ctdb-1.13-7-gaf72f8b

2012-03-14 Thread Ronnie Sahlberg
The branch, master has been updated
   via  af72f8bc05f78983f9624d81788f9d89af474b11 (commit)
   via  a7ad2fb75f06791508dd928d2a0c305fc7f7b814 (commit)
   via  db69a39ca010b2c8853b13466e30792dda350cdb (commit)
   via  a0340a50c2acd9ccc281faef032a364254f7f95a (commit)
  from  f06634951331232cddf0b48eac3552b92aca5b93 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=master


- Log -
commit af72f8bc05f78983f9624d81788f9d89af474b11
Merge: f06634951331232cddf0b48eac3552b92aca5b93 
a7ad2fb75f06791508dd928d2a0c305fc7f7b814
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Thu Mar 15 16:09:41 2012 +1100

Merge remote branch 'martins/master'

commit a7ad2fb75f06791508dd928d2a0c305fc7f7b814
Author: Martin Schwenke mar...@meltin.net
Date:   Mon Feb 13 13:34:57 2012 +1100

Tests - functions/environment - set and use $CTDB_NODES

ctdb listnodes changed so that it never tries to contact the daemon
but reads the local nodes file instead.  This fails if the nodes file
is in a non-default place but $CTDB_NODES isn't set.

Signed-off-by: Martin Schwenke mar...@meltin.net

commit db69a39ca010b2c8853b13466e30792dda350cdb
Author: Martin Schwenke mar...@meltin.net
Date:   Mon Feb 13 13:32:25 2012 +1100

Tests - simple - make disconnected node tests handle changed error message

Recent changes to nodestring parsing generalised the error message
logged when trying to talk to a disconnected node.

Signed-off-by: Martin Schwenke mar...@meltin.net

commit a0340a50c2acd9ccc281faef032a364254f7f95a
Author: Martin Schwenke mar...@meltin.net
Date:   Mon Feb 13 13:31:18 2012 +1100

ctdb tool: fix thinko in nodestatus command output

If used with -n nodes the current node needs to change.

Signed-off-by: Martin Schwenke mar...@meltin.net

---

Summary of changes:
 tests/scripts/ctdb_test_env|2 ++
 tests/scripts/ctdb_test_functions.bash |8 +++-
 tests/simple/09_ctdb_ping.sh   |2 +-
 ..._ctdb_config_check_error_on_unreachable_ctdb.sh |6 +++---
 tools/ctdb.c   |2 +-
 5 files changed, 10 insertions(+), 10 deletions(-)


Changeset truncated at 500 lines:

diff --git a/tests/scripts/ctdb_test_env b/tests/scripts/ctdb_test_env
index 5c3006e..f550d79 100755
--- a/tests/scripts/ctdb_test_env
+++ b/tests/scripts/ctdb_test_env
@@ -32,6 +32,8 @@ if [ ! -n $CTDB_TEST_REAL_CLUSTER ] ; then
 done
 
 PATH=${CTDB_DIR}/bin:${CTDB_DIR}/tests/bin:${PATH}
+
+export CTDB_NODES=$var_dir/nodes.txt
 fi
 
 # If $VALGRIND is set then use it whenever ctdb is called, but only if
diff --git a/tests/scripts/ctdb_test_functions.bash 
b/tests/scripts/ctdb_test_functions.bash
index 670b91e..256ad99 100644
--- a/tests/scripts/ctdb_test_functions.bash
+++ b/tests/scripts/ctdb_test_functions.bash
@@ -720,10 +720,9 @@ daemons_setup ()
 
 mkdir -p $var_dir/test.db/persistent
 
-local nodes=$var_dir/nodes.txt
 local public_addresses=$var_dir/public_addresses.txt
 local no_public_addresses=$var_dir/no_public_addresses.txt
-rm -f $nodes $public_addresses $no_public_addresses
+rm -f $CTDB_NODES $public_addresses $no_public_addresses
 
 # If there are (strictly) greater than 2 nodes then we'll randomly
 # choose a node to have no public addresses.
@@ -737,7 +736,7 @@ daemons_setup ()
echo ::$i  $nodes
ip addr add ::$i/128 dev lo
else
-   echo 127.0.0.$i  $nodes
+   echo 127.0.0.$i  $CTDB_NODES
# 2 public addresses on most nodes, just to make things interesting.
if [ $(($i - 1)) -ne $no_public_ips ] ; then
echo 192.0.2.$i/24 lo  $public_addresses
@@ -754,7 +753,6 @@ daemons_start_1 ()
 
 local var_dir=$CTDB_DIR/tests/var
 
-local nodes=$var_dir/nodes.txt
 local public_addresses=$var_dir/public_addresses.txt
 local no_public_addresses=$var_dir/no_public_addresses.txt
 
@@ -765,7 +763,7 @@ daemons_start_1 ()
echo Node $no_public_ips will have no public IPs.
 fi
 
-local ctdb_options=--reclock=$var_dir/rec.lock --nlist $nodes 
--nopublicipcheck --event-script-dir=$CTDB_DIR/tests/events.d 
--logfile=$var_dir/daemons.log -d 0 --dbdir=$var_dir/test.db 
--dbdir-persistent=$var_dir/test.db/persistent 
--dbdir-state=$var_dir/test.db/state
+local ctdb_options=--reclock=$var_dir/rec.lock --nlist $CTDB_NODES 
--nopublicipcheck --event-script-dir=$CTDB_DIR/tests/events.d 
--logfile=$var_dir/daemons.log -d 0 --dbdir=$var_dir/test.db 
--dbdir-persistent=$var_dir/test.db/persistent 
--dbdir-state=$var_dir/test.db/state
 
 if [ $(id -u) -eq 0 ]; then
 ctdb_options=$ctdb_options --public-interface=lo
diff --git a/tests/simple/09_ctdb_ping.sh b/tests/simple

[SCM] CTDB repository - branch master updated - ctdb-1.13-3-gf066349

2012-03-12 Thread Ronnie Sahlberg
The branch, master has been updated
   via  f06634951331232cddf0b48eac3552b92aca5b93 (commit)
  from  68390cb1e8c7f30be034b8ca6c9215b40147305b (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=master


- Log -
commit f06634951331232cddf0b48eac3552b92aca5b93
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Mon Mar 12 16:09:32 2012 +1100

Test: add test tool and test that UPDATE_RECORD works on persistent 
databases

---

Summary of changes:
 Makefile.in|5 +
 tests/simple/72_update_record_persistent.sh|   89 +++
 ...only_loop.c = ctdb_update_record_persistent.c} |   91 +---
 3 files changed, 136 insertions(+), 49 deletions(-)
 create mode 100755 tests/simple/72_update_record_persistent.sh
 copy tests/src/{ctdb_fetch_readonly_loop.c = ctdb_update_record_persistent.c} 
(57%)


Changeset truncated at 500 lines:

diff --git a/Makefile.in b/Makefile.in
index 24283f3..9055a8f 100755
--- a/Makefile.in
+++ b/Makefile.in
@@ -85,6 +85,7 @@ TEST_BINS=tests/bin/ctdb_bench tests/bin/ctdb_fetch 
tests/bin/ctdb_fetch_one \
tests/bin/ctdb_randrec tests/bin/ctdb_persistent \
tests/bin/ctdb_traverse tests/bin/rb_test tests/bin/ctdb_transaction \
tests/bin/ctdb_takeover_tests tests/bin/ctdb_update_record \
+   tests/bin/ctdb_update_record_persistent \
tests/bin/ctdb_tool_libctdb tests/bin/ctdb_tool_stubby \
@INFINIBAND_BINS@
 
@@ -215,6 +216,10 @@ tests/bin/ctdb_update_record: $(CTDB_CLIENT_OBJ) 
tests/src/ctdb_update_record.o
@echo Linking $@
@$(CC) $(CFLAGS) -o $@ tests/src/ctdb_update_record.o 
$(CTDB_CLIENT_OBJ) $(LIB_FLAGS)
 
+tests/bin/ctdb_update_record_persistent: $(CTDB_CLIENT_OBJ) 
tests/src/ctdb_update_record_persistent.o 
+   @echo Linking $@
+   @$(CC) $(CFLAGS) -o $@ tests/src/ctdb_update_record_persistent.o 
$(CTDB_CLIENT_OBJ) $(LIB_FLAGS)
+
 tests/bin/ctdb_store: $(CTDB_CLIENT_OBJ) tests/src/ctdb_store.o 
@echo Linking $@
@$(CC) $(CFLAGS) -o $@ tests/src/ctdb_store.o $(CTDB_CLIENT_OBJ) 
$(LIB_FLAGS)
diff --git a/tests/simple/72_update_record_persistent.sh 
b/tests/simple/72_update_record_persistent.sh
new file mode 100755
index 000..d232b40
--- /dev/null
+++ b/tests/simple/72_update_record_persistent.sh
@@ -0,0 +1,89 @@
+#!/bin/bash
+
+test_info()
+{
+cat EOF
+UPDATE_RECORD control should be able to create new records and update
+existing records in a persistent database.
+
+Prerequisites:
+
+* An active CTDB cluster with at least one active node.
+
+Steps:
+
+1. Verify that the status on all of the ctdb nodes is 'OK'.
+2. create a persistent test database
+3, wipe the database to make sure it is empty
+4, create a new record
+5, update the record
+
+Expected results:
+
+* 4 created record found in the tdb
+* 5 updated record found in the tdb
+
+EOF
+}
+
+. ctdb_test_functions.bash
+
+ctdb_test_init $@
+
+set -e
+
+cluster_is_healthy
+
+try_command_on_node 0 $CTDB listnodes
+num_nodes=$(echo $out | wc -l)
+
+# create a temporary persistent database to test with
+echo create persistent test database persistent_test.tdb
+try_command_on_node -q 0 $CTDB_TEST_WRAPPER ctdb attach persistent_test.tdb 
persistent
+
+
+# 3,
+echo wipe the persistent test database
+try_command_on_node -q 0 $CTDB_TEST_WRAPPER ctdb wipedb persistent_test.tdb
+echo force a recovery
+try_command_on_node -q 0 $CTDB_TEST_WRAPPER ctdb recover
+
+# check that the database is wiped
+num_records=$(try_command_on_node -v -pq 1 $CTDB_TEST_WRAPPER ctdb cattdb 
persistent_test.tdb | grep key | wc -l)
+[ $num_records != 0 ]  {
+echo BAD: we did not end up with an empty database
+exit 1
+}
+echo OK. database was wiped
+
+TDB=`try_command_on_node -v -q 0 $CTDB_TEST_WRAPPER ctdb getdbmap | grep 
persistent_test.tdb | sed -e s/.*path:// -e s/ .*//`
+
+# 4,
+echo Create a new record in the persistent database using UPDATE_RECORD
+try_command_on_node -q 0 $CTDB_TEST_WRAPPER ctdb_update_record_persistent  
--database=persistent_test.tdb --record=Update_Record_Persistent 
--value=FirstValue
+
+num_records=`tdbdump $TDB | grep FirstValue | wc -l`
+[ $num_records != 1 ]  {
+echo BAD: we did find the record after the create/update
+exit 1
+}
+
+# 5,
+echo Modify an existing record in the persistent database using UPDATE_RECORD
+try_command_on_node -q 0 $CTDB_TEST_WRAPPER ctdb_update_record_persistent  
--database=persistent_test.tdb --record=Update_Record_Persistent 
--value=SecondValue
+
+num_records=`tdbdump $TDB | grep FirstValue | wc -l`
+[ $num_records != 0 ]  {
+echo BAD: we still found the old record after the modify/update
+exit 1
+}
+
+num_records=`tdbdump $TDB | grep SecondValue | wc -l`
+[ $num_records != 1 ]  {
+echo BAD: could not find the record after the modify/update
+exit 1

[SCM] CTDB repository - branch 1.13 updated - ctdb-1.13-2-g8d606e5

2012-03-11 Thread Ronnie Sahlberg
The branch, 1.13 has been updated
   via  8d606e5f71072ccc60bcac54dfd9e92bb699c6e5 (commit)
  from  0c0e887912ca37682312091f4b89ff5167b254d9 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=1.13


- Log -
commit 8d606e5f71072ccc60bcac54dfd9e92bb699c6e5
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Mon Mar 12 13:13:51 2012 +1100

ReadOnly: Only restrict the readonly sanity/paranoia check that only allows
UPDATE_RECORD to create a record to only apply to normal databases, not
persistent databases or else the net registry command breaks
and other persistent database creation/updates .

---

Summary of changes:
 server/ctdb_update_record.c |5 -
 1 files changed, 4 insertions(+), 1 deletions(-)


Changeset truncated at 500 lines:

diff --git a/server/ctdb_update_record.c b/server/ctdb_update_record.c
index f8be8e8..a4a6fc4 100644
--- a/server/ctdb_update_record.c
+++ b/server/ctdb_update_record.c
@@ -325,7 +325,10 @@ int32_t ctdb_control_update_record(struct ctdb_context 
*ctdb,
state-ctdb_db = ctdb_db;
state-c   = c;
state-m   = m;
-   state-flags   = UPDATE_FLAGS_REPLACE_ONLY;
+   state-flags   = 0;
+   if (!ctdb_db-persistent) {
+   state-flags   = UPDATE_FLAGS_REPLACE_ONLY;
+   }
 
/* create a child process to take out a transaction and
   write the data.


-- 
CTDB repository


[SCM] CTDB repository - branch 1.2.40 updated - ctdb-1.9.1-557-gd967d1b

2012-03-11 Thread Ronnie Sahlberg
The branch, 1.2.40 has been updated
   via  d967d1bcf72d32312ca765bb467d03ddd4690cb7 (commit)
   via  ba1b678db4dde637d7aba1037f446038d2093ec2 (commit)
   via  10249a8f350471d5c6e0bb16a7c1276ec3bcc22b (commit)
  from  9fd1d9cca3fcced1aa4af5d515f7e2617eae4a50 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=1.2.40


- Log -
commit d967d1bcf72d32312ca765bb467d03ddd4690cb7
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Mon Mar 12 13:05:07 2012 +1100

new version 1.2.42

commit ba1b678db4dde637d7aba1037f446038d2093ec2
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Mon Mar 12 13:03:19 2012 +1100

ReadOnly: Only restrict the readonly sanity/paranoia check that only allows
UPDATE_RECORD to create a record to only apply to normal databases, not
persistent databases or else the net registry command breaks
and other persistent database creation/updates .

S1035666

commit 10249a8f350471d5c6e0bb16a7c1276ec3bcc22b
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Tue Mar 6 16:03:18 2012 +1100

new version 1.2.41

---

Summary of changes:
 packaging/RPM/ctdb.spec.in |   10 +-
 server/ctdb_persistent.c   |5 -
 2 files changed, 13 insertions(+), 2 deletions(-)


Changeset truncated at 500 lines:

diff --git a/packaging/RPM/ctdb.spec.in b/packaging/RPM/ctdb.spec.in
index 3027baf..7519be1 100644
--- a/packaging/RPM/ctdb.spec.in
+++ b/packaging/RPM/ctdb.spec.in
@@ -3,7 +3,7 @@ Name: ctdb
 Summary: Clustered TDB
 Vendor: Samba Team
 Packager: Samba Team sa...@samba.org
-Version: 1.2.40
+Version: 1.2.42
 Release: 1GITHASH
 Epoch: 0
 License: GNU GPL version 3
@@ -144,6 +144,14 @@ development libraries for ctdb
 %{_libdir}/libctdb.a
 
 %changelog
+* Mon Mar 12 2012 : Version 1.2.42
+ - Allow non-readonly database UPDATE_RECORD to create non-existing records
+   or else net command and the registry breaks.
+   S1035666
+* Tue Mar 6 2012 : Version 1.2.41
+ - rewrite of 13.per_ip_routing script
+ - readonly record bugfixes
+ - add statistics for readonly delegations and revokes
 * Mon Feb 6 2012 : Version 1.2.40
  - Fix for LCP2 failure to balance uneven netgroups
  - Add ReadOnly record support
diff --git a/server/ctdb_persistent.c b/server/ctdb_persistent.c
index 669021a..01c0e86 100644
--- a/server/ctdb_persistent.c
+++ b/server/ctdb_persistent.c
@@ -732,7 +732,10 @@ int32_t ctdb_control_update_record(struct ctdb_context 
*ctdb,
state-ctdb_db = ctdb_db;
state-c   = c;
state-m   = m;
-   state-flags   = UPDATE_FLAGS_REPLACE_ONLY;
+   state-flags   = 0;
+   if (!ctdb_db-persistent) {
+   state-flags   = UPDATE_FLAGS_REPLACE_ONLY;
+   }
 
/* create a child process to take out a transaction and 
   write the data.


-- 
CTDB repository


[SCM] CTDB repository - branch master updated - ctdb-1.13-2-g68390cb

2012-03-11 Thread Ronnie Sahlberg
The branch, master has been updated
   via  68390cb1e8c7f30be034b8ca6c9215b40147305b (commit)
  from  fb00e1290fcea3386132a46c883994019a43799a (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=master


- Log -
commit 68390cb1e8c7f30be034b8ca6c9215b40147305b
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Mon Mar 12 13:13:51 2012 +1100

ReadOnly: Only restrict the readonly sanity/paranoia check that only allows
UPDATE_RECORD to create a record to only apply to normal databases, not
persistent databases or else the net registry command breaks
and other persistent database creation/updates .

---

Summary of changes:
 server/ctdb_update_record.c |5 -
 1 files changed, 4 insertions(+), 1 deletions(-)


Changeset truncated at 500 lines:

diff --git a/server/ctdb_update_record.c b/server/ctdb_update_record.c
index f8be8e8..a4a6fc4 100644
--- a/server/ctdb_update_record.c
+++ b/server/ctdb_update_record.c
@@ -325,7 +325,10 @@ int32_t ctdb_control_update_record(struct ctdb_context 
*ctdb,
state-ctdb_db = ctdb_db;
state-c   = c;
state-m   = m;
-   state-flags   = UPDATE_FLAGS_REPLACE_ONLY;
+   state-flags   = 0;
+   if (!ctdb_db-persistent) {
+   state-flags   = UPDATE_FLAGS_REPLACE_ONLY;
+   }
 
/* create a child process to take out a transaction and
   write the data.


-- 
CTDB repository


[SCM] CTDB repository - branch 1.2.40 updated - ctdb-1.9.1-554-g9fd1d9c

2012-03-10 Thread Ronnie Sahlberg
The branch, 1.2.40 has been updated
   via  9fd1d9cca3fcced1aa4af5d515f7e2617eae4a50 (commit)
   via  0b8280258f0d228b2e9659dccfd9633321323aaa (commit)
   via  f670f660a781b944aa776bd6270c4007b140cdd3 (commit)
   via  c02ee2bf6d83c8611ccb4e9b1036440f1231b724 (commit)
   via  adb90f1e80b2d34233b8175e77515c589f4026f5 (commit)
  from  545c343b19258fce01562b15f274eaf1a1deafc8 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=1.2.40


- Log -
commit 9fd1d9cca3fcced1aa4af5d515f7e2617eae4a50
Author: Martin Schwenke mar...@meltin.net
Date:   Tue Aug 23 16:32:34 2011 +1000

Eventscripts - new function ctdb_check_args()

Pass this $@ to do common eventscript argument checking.

For regular use putting this in 00.ctdb would be enough.  However, for
developer testing it can be useful to call this in other eventscripts.
For example, 10.interfaces and 13.per_ip_routing currently check these
by hand.

Signed-off-by: Martin Schwenke mar...@meltin.net

commit 0b8280258f0d228b2e9659dccfd9633321323aaa
Author: Martin Schwenke mar...@meltin.net
Date:   Tue Mar 6 11:21:41 2012 +1100

Eventscript functions - remove functions only used by 13.per_ip_routing

The relevant functions are now in that script.

Signed-off-by: Martin Schwenke mar...@meltin.net

commit f670f660a781b944aa776bd6270c4007b140cdd3
Author: Martin Schwenke mar...@meltin.net
Date:   Thu Mar 1 15:23:53 2012 +1100

Eventscripts - redesign and rewrite 13.per_ip_routing

The current version is quite difficult to read.  This one is hopefully
clearer.

Major changes:

* The configuration file has a more forgiving syntax.  Items can be
  separated by arbitrary whitespace.

* Mappings between IP addresses and table IDs are no longer stored in
  files in a state directory.  Instead they are stored in
  /etc/iproute2/rt_tables as mappings between table IDs and labels, as
  allowed by the ip command.  The current structure of the labels is
  ctdb.source-ip.  This means that once the labels are setup the
  routing tables can be referenced by just knowing the source IP.  As
  with the old state directory, mappings in this file owned by CTDB
  are deleted when CTDB shuts down.

* There are no release or re-add scripts.

  - Release scripts are not necessary as an optimisation because of
the previous improvement (i.e. use of rt_tables).  No lookup is
necessary to delete rules or flush tables.

  - Re-add scripts are no longer used.  Routes can still go missing
when removal of a primary IP from an interfaces (or similar)
causes removal of all other addresses (i.e. secondaries) and also
all associated routes.  However, any missing routes are now
re-added in the ipreallocated event.  This happens shortly after
takeip/releaseip/updateip and means that the routes will only be
re-added once.  The window for missing routes is slightly bigger
but is not expected to be significant.

* The magic __auto_link_local__ configuration value no longer causes
  a dynamic configuration file to be maintained in a state directory.
  The link local configuration is now generated when needed from the
  public_addresses file.  This greatly simplifies the code.  This
  approach is slightly less efficient but should not be significant.

The above changes mean that, apart from maintaining mappings in the
rt_tables file, there are no state files kept anymore.

Some utility functions only used by this script have been rewritten
and moved into this script.  They will be removed from the functions
file by a future commit.

The route re-add code will also be removed from interface_modify.sh by
a future commit.  It is currently harmless.

Signed-off-by: Martin Schwenke mar...@meltin.net

commit c02ee2bf6d83c8611ccb4e9b1036440f1231b724
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Tue Mar 6 10:51:00 2012 +1100

readonly, dont allow any kind of fast vacuum for ro delegated records

commit adb90f1e80b2d34233b8175e77515c589f4026f5
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Fri Mar 2 14:48:15 2012 +1100

Revert READONLY: readonly fetch collapse. Make sure we only keep one 
single readonly fetch for a record in flight at a time.

This reverts commit 9bde066f6eb46124168e5686fc41a323e67401e8.

---

Summary of changes:
 config/events.d/13.per_ip_routing |  663 ++---
 config/functions  |   84 ++
 include/ctdb_private.h|4 -
 server/ctdb_call.c|3 +-
 server/ctdb_daemon.c  |  208 
 server/ctdb_ltdb_server.c

[SCM] CTDB repository - branch 1.2.39-28 created - ctdb-1.9.1-497-g635ff89

2012-03-10 Thread Ronnie Sahlberg
The branch, 1.2.39-28 has been created
at  635ff897405735aaf856f5dc900c2ae6aacf9243 (commit)

- Log -
commit 635ff897405735aaf856f5dc900c2ae6aacf9243
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Wed Mar 7 17:05:37 2012 +1100

new test version

commit 6eab12c91082594dc5a4291e66d10112e5da131e
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Wed Mar 7 17:02:41 2012 +1100

add max hop count buckets to see how bad hopcounts are

commit 9e702ae900809bde8c3ba7e7a29a1171d0b7ed86
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Wed Mar 7 16:32:19 2012 +1100

set max lacount to default to 20

commit 0c4ae347a20a780fe0326036de699e9f6cbb6d26
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Wed Mar 7 16:11:50 2012 +1100

Add back la-count based migration

commit 8bc2cca22faf33dad8908567fa30b55173cb3426
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Wed Mar 7 15:58:51 2012 +1100

Record Fetch Collapse: Collapse multiple fetch request into one single 
request

When multiple clients fetch the same record concurrently, send only one 
single
fetch across the network and deferr all other fetches locally.
This improves performance for hot records and reduces cpu load on ctdb.

commit 39d9e8b813e5957bd8319bd5ad6c4c9978c27dad
Author: Volker Lendecke v...@samba.org
Date:   Mon Oct 31 13:29:13 2011 +0100

Add CTDB_CONTROL_CHECK_SRVID

commit c1ac43506f32f92c4b2a5267dbf87b2dbe74a5bf
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Wed Feb 29 12:25:41 2012 +1100

Vacuuming: change default timeout to 120 seconds

S1035431

---


-- 
CTDB repository


[SCM] CTDB repository - branch 1.2.40 updated - ctdb-1.9.1-546-g0a52799

2012-03-01 Thread Ronnie Sahlberg
The branch, 1.2.40 has been updated
   via  0a52799f85de9c9dc0ac8ae62e7f2829a30eb8bb (commit)
   via  ce57fcab99fa13548ae3693f471c7ecde08f67f3 (commit)
   via  dbf2c5b25833bf05cc3d1b9a9ee2186143386f35 (commit)
   via  1083ef705cb67185f6c199c01850e839d4f3c8bb (commit)
  from  fd33e6ff1e349e3d6d1d2e78ab14942c97aba731 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=1.2.40


- Log -
commit 0a52799f85de9c9dc0ac8ae62e7f2829a30eb8bb
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Fri Mar 2 09:43:39 2012 +1100

READONLY: when updating a remote node to revoke a delegation, make sure
we dont create thje record if it doesnt exist

commit ce57fcab99fa13548ae3693f471c7ecde08f67f3
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Mon Feb 13 10:27:59 2012 +1100

READONLY:  allow specifying the db name for setdbreadonly instead of just 
the hash

commit dbf2c5b25833bf05cc3d1b9a9ee2186143386f35
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Fri Mar 2 09:04:39 2012 +1100

Niceify the readonlyrecord API. Dont force clients to be exposed to the 
fetch_with_header function

We dont strictly need to force clients to use CTDB_FETCH_WITH_HEADER 
instead of CTDB_FETCH when they ask for readonly records.
Have ctdbd internally remap this internally to FETCH_WITH_HEADER and map 
the reply back to CTDB_FETCH_FUNC or CTDB_FETCH_WITH_HEADER_FUNC based on what 
the client initially asked for.

This removes the need for the client to know about the 
CTDB_FETCH_WITH_HEADER_FUNC function and simplifies the client code.
Clients that do not care what the header after the request is can just 
continue using the old CTDB_FETCH_FUNC call and ctdbd will do all the difficult 
stuff.

commit 1083ef705cb67185f6c199c01850e839d4f3c8bb
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Fri Mar 2 08:53:16 2012 +1100

READONLY: skip vacuuming or deleting records with readonly delegations.
these records are hot. wait until they have been revoked before we recall 
them.

---

Summary of changes:
 doc/readonlyrecords.txt  |7 +--
 libctdb/ctdb.c   |8 
 server/ctdb_call.c   |4 
 server/ctdb_daemon.c |   40 ++--
 server/ctdb_persistent.c |   17 +
 server/ctdb_recover.c|   14 ++
 server/ctdb_vacuum.c |5 +
 tools/ctdb.c |   33 +++--
 8 files changed, 118 insertions(+), 10 deletions(-)


Changeset truncated at 500 lines:

diff --git a/doc/readonlyrecords.txt b/doc/readonlyrecords.txt
index acdab2e..f8f1095 100644
--- a/doc/readonlyrecords.txt
+++ b/doc/readonlyrecords.txt
@@ -89,7 +89,10 @@ This new database is used for tracking delegations for the 
records. A record in
 This tracking database is lockless, using TDB_NOLOCK, and is only ever 
accessed by the main ctdbd daemon.
 The lockless nature and the fact that no other process ever access this TDB 
means we are guranteed non-blocking access to records in the trcking database.
 
-The ctdb_call PDU is allocated with two new flags WANT_READONLY and 
WITH_HEADER.
+The ctdb_call PDU is allocated with a new flags WANT_READONLY and possibly 
also a new callid: CTDB_FETCH_WITH_HEADER_FUNC.
+This new function returns not only the record, as CTDB_FETCH_FUNC does, but 
also returns the HEADER prepended to the record.
+This function is optional, clients that do not care what the header is can 
continue using just CTDB_FETCH_FUNC
+
 This first flag is used to explicitely requesting a read-only record from the 
DMASTER/LMASTER.
 The second flag is used to request that the fetch operation will return not 
only the data for the record but also
 the record header. 
@@ -137,7 +140,7 @@ This will change to instead do
 goto finished
 else
 unlock record 
-ask ctdb for read-only copy (WANT_READONLY|WITH_HEADER)
+ask ctdb for read-only copy (WANT_READONLY[|WITH_HEADER])
 if failed to get read-only copy (*A)
 ask ctdb to migrate the record onto the node
 goto try_again
diff --git a/libctdb/ctdb.c b/libctdb/ctdb.c
index 36cc113..46f4953 100644
--- a/libctdb/ctdb.c
+++ b/libctdb/ctdb.c
@@ -819,13 +819,13 @@ static void readrecordlock_retry(struct ctdb_connection 
*ctdb,
struct ctdb_reply_call *reply;
TDB_DATA data;
 
-   /* OK, we've received reply to fetch-with-header migration */
-   reply = unpack_reply_call(ctdb, req, CTDB_FETCH_WITH_HEADER_FUNC);
+   /* OK, we've received reply to fetch migration */
+   reply = unpack_reply_call(ctdb, req, CTDB_FETCH_FUNC);
if (!reply || reply-status != 0) {
if (reply) {
DEBUG(ctdb

[SCM] CTDB repository - branch 1.2.40 updated - ctdb-1.9.1-547-gc51154b

2012-03-01 Thread Ronnie Sahlberg
The branch, 1.2.40 has been updated
   via  c51154b79be94198324c321037045bb85cd9 (commit)
  from  0a52799f85de9c9dc0ac8ae62e7f2829a30eb8bb (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=1.2.40


- Log -
commit c51154b79be94198324c321037045bb85cd9
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Fri Mar 2 10:52:00 2012 +1100

READONLY: only send a control to schedule fast-vacuuming from child context 
iff we have a connection open to the main daemon

there are some child processes where we do not create a connection to the 
main daemon (switch_from_server_to_client()) because it is expensive to set up 
and we normally might not need to talk to the daemon at all via a domainsocket.
but we might want to still call to ctdb_ltdb_store() from such chil 
processes.

---

Summary of changes:
 client/ctdb_client.c   |4 +++-
 common/ctdb_util.c |1 +
 include/ctdb_private.h |3 +++
 server/ctdb_vacuum.c   |8 
 4 files changed, 15 insertions(+), 1 deletions(-)


Changeset truncated at 500 lines:

diff --git a/client/ctdb_client.c b/client/ctdb_client.c
index 487989c..8b9df42 100644
--- a/client/ctdb_client.c
+++ b/client/ctdb_client.c
@@ -4087,7 +4087,9 @@ int switch_from_server_to_client(struct ctdb_context 
*ctdb, const char *fmt, ...
return -1;
}
 
-return 0;
+   ctdb-can_send_controls = true;
+
+   return 0;
 }
 
 /*
diff --git a/common/ctdb_util.c b/common/ctdb_util.c
index 1ff4c1f..061c16d 100644
--- a/common/ctdb_util.c
+++ b/common/ctdb_util.c
@@ -346,6 +346,7 @@ pid_t ctdb_fork(struct ctdb_context *ctdb)
if (ctdb-do_setsched) {
ctdb_restore_scheduler(ctdb);
}
+   ctdb-can_send_controls = false;
}
return pid;
 }
diff --git a/include/ctdb_private.h b/include/ctdb_private.h
index 7d0a6d8..8180722 100644
--- a/include/ctdb_private.h
+++ b/include/ctdb_private.h
@@ -499,6 +499,9 @@ struct ctdb_context {
 
/* list of event script callback functions that are active */
struct event_script_callback *script_callbacks;
+
+   /* if we are a child process, do we have a domain socket to send 
controls on */
+   bool can_send_controls;
 };
 
 struct ctdb_db_context {
diff --git a/server/ctdb_vacuum.c b/server/ctdb_vacuum.c
index 181393b..bb71be7 100644
--- a/server/ctdb_vacuum.c
+++ b/server/ctdb_vacuum.c
@@ -1371,6 +1371,14 @@ int32_t ctdb_local_schedule_for_deletion(struct 
ctdb_db_context *ctdb_db,
return ret;
}
 
+   /* if we dont have a connection to the daemon we can not send
+  a control. For example sometimes from update_record control child
+  process.
+   */
+   if (!ctdb_db-ctdb-can_send_controls) {
+   return -1;
+   }
+
/* child process: send the main daemon a control */
 
indata.dsize = offsetof(struct ctdb_control_schedule_for_deletion, key) 
+ key.dsize;


-- 
CTDB repository


[SCM] CTDB repository - branch 1.13 updated - ctdb-1.13-1-g0c0e887

2012-03-01 Thread Ronnie Sahlberg
The branch, 1.13 has been updated
   via  0c0e887912ca37682312091f4b89ff5167b254d9 (commit)
  from  05e75dea9eaa0decc2909e0c2cf1715c81374200 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=1.13


- Log -
commit 0c0e887912ca37682312091f4b89ff5167b254d9
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Fri Mar 2 12:57:23 2012 +1100

READONLY: when updating a remote node to revoke a delegation, make sure we 
dont create the record if it doesnt already exist

---

Summary of changes:
 server/ctdb_call.c  |1 +
 server/ctdb_update_record.c |   17 +
 2 files changed, 18 insertions(+), 0 deletions(-)


Changeset truncated at 500 lines:

diff --git a/server/ctdb_call.c b/server/ctdb_call.c
index 9922233..50f4cb2 100644
--- a/server/ctdb_call.c
+++ b/server/ctdb_call.c
@@ -1300,6 +1300,7 @@ int ctdb_start_revoke_ro_record(struct ctdb_context 
*ctdb, struct ctdb_db_contex
int ret;
 
header-flags = 
~(CTDB_REC_RO_REVOKING_READONLY|CTDB_REC_RO_HAVE_DELEGATIONS|CTDB_REC_RO_HAVE_READONLY);
+   header-flags |= CTDB_REC_FLAG_MIGRATED_WITH_DATA;
header-rsn   -= 1;
 
if ((rc = talloc_zero(ctdb_db, struct revokechild_handle)) == NULL) {
diff --git a/server/ctdb_update_record.c b/server/ctdb_update_record.c
index 4413597..f8be8e8 100644
--- a/server/ctdb_update_record.c
+++ b/server/ctdb_update_record.c
@@ -28,8 +28,11 @@ struct ctdb_persistent_write_state {
struct ctdb_db_context *ctdb_db;
struct ctdb_marshall_buffer *m;
struct ctdb_req_control *c;
+   uint32_t flags;
 };
 
+/* dont create/update records that does not exist locally */
+#define UPDATE_FLAGS_REPLACE_ONLY  1
 
 /*
   called from a child process to write the data
@@ -62,6 +65,19 @@ static int ctdb_persistent_store(struct 
ctdb_persistent_write_state *state)
goto failed;
}
 
+   /* we must check if the record exists or not because
+  ctdb_ltdb_fetch will unconditionally create a record
+*/
+   if (state-flags  UPDATE_FLAGS_REPLACE_ONLY) {
+   TDB_DATA rec;
+   rec = tdb_fetch(state-ctdb_db-ltdb-tdb, key);
+   if (rec.dsize == 0) {
+   talloc_free(tmp_ctx);
+   continue;
+   }
+   free(rec.dptr);
+   }
+
/* fetch the old header and ensure the rsn is less than the new 
rsn */
ret = ctdb_ltdb_fetch(state-ctdb_db, key, oldheader, tmp_ctx, 
olddata);
if (ret != 0) {
@@ -309,6 +325,7 @@ int32_t ctdb_control_update_record(struct ctdb_context 
*ctdb,
state-ctdb_db = ctdb_db;
state-c   = c;
state-m   = m;
+   state-flags   = UPDATE_FLAGS_REPLACE_ONLY;
 
/* create a child process to take out a transaction and
   write the data.


-- 
CTDB repository


[SCM] CTDB repository - branch master updated - ctdb-1.13-1-gfb00e12

2012-03-01 Thread Ronnie Sahlberg
The branch, master has been updated
   via  fb00e1290fcea3386132a46c883994019a43799a (commit)
  from  05e75dea9eaa0decc2909e0c2cf1715c81374200 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=master


- Log -
commit fb00e1290fcea3386132a46c883994019a43799a
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Fri Mar 2 12:57:23 2012 +1100

READONLY: when updating a remote node to revoke a delegation, make sure we 
dont create the record if it doesnt already exist

---

Summary of changes:
 server/ctdb_call.c  |1 +
 server/ctdb_update_record.c |   17 +
 2 files changed, 18 insertions(+), 0 deletions(-)


Changeset truncated at 500 lines:

diff --git a/server/ctdb_call.c b/server/ctdb_call.c
index 9922233..50f4cb2 100644
--- a/server/ctdb_call.c
+++ b/server/ctdb_call.c
@@ -1300,6 +1300,7 @@ int ctdb_start_revoke_ro_record(struct ctdb_context 
*ctdb, struct ctdb_db_contex
int ret;
 
header-flags = 
~(CTDB_REC_RO_REVOKING_READONLY|CTDB_REC_RO_HAVE_DELEGATIONS|CTDB_REC_RO_HAVE_READONLY);
+   header-flags |= CTDB_REC_FLAG_MIGRATED_WITH_DATA;
header-rsn   -= 1;
 
if ((rc = talloc_zero(ctdb_db, struct revokechild_handle)) == NULL) {
diff --git a/server/ctdb_update_record.c b/server/ctdb_update_record.c
index 4413597..f8be8e8 100644
--- a/server/ctdb_update_record.c
+++ b/server/ctdb_update_record.c
@@ -28,8 +28,11 @@ struct ctdb_persistent_write_state {
struct ctdb_db_context *ctdb_db;
struct ctdb_marshall_buffer *m;
struct ctdb_req_control *c;
+   uint32_t flags;
 };
 
+/* dont create/update records that does not exist locally */
+#define UPDATE_FLAGS_REPLACE_ONLY  1
 
 /*
   called from a child process to write the data
@@ -62,6 +65,19 @@ static int ctdb_persistent_store(struct 
ctdb_persistent_write_state *state)
goto failed;
}
 
+   /* we must check if the record exists or not because
+  ctdb_ltdb_fetch will unconditionally create a record
+*/
+   if (state-flags  UPDATE_FLAGS_REPLACE_ONLY) {
+   TDB_DATA rec;
+   rec = tdb_fetch(state-ctdb_db-ltdb-tdb, key);
+   if (rec.dsize == 0) {
+   talloc_free(tmp_ctx);
+   continue;
+   }
+   free(rec.dptr);
+   }
+
/* fetch the old header and ensure the rsn is less than the new 
rsn */
ret = ctdb_ltdb_fetch(state-ctdb_db, key, oldheader, tmp_ctx, 
olddata);
if (ret != 0) {
@@ -309,6 +325,7 @@ int32_t ctdb_control_update_record(struct ctdb_context 
*ctdb,
state-ctdb_db = ctdb_db;
state-c   = c;
state-m   = m;
+   state-flags   = UPDATE_FLAGS_REPLACE_ONLY;
 
/* create a child process to take out a transaction and
   write the data.


-- 
CTDB repository


[SCM] CTDB repository - branch 1.2.40 updated - ctdb-1.9.1-549-g545c343

2012-03-01 Thread Ronnie Sahlberg
The branch, 1.2.40 has been updated
   via  545c343b19258fce01562b15f274eaf1a1deafc8 (commit)
   via  9bde066f6eb46124168e5686fc41a323e67401e8 (commit)
  from  c51154b79be94198324c321037045bb85cd9 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=1.2.40


- Log -
commit 545c343b19258fce01562b15f274eaf1a1deafc8
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Wed Feb 8 13:42:30 2012 +1100

STATISTICS: add total counts for number of delegations and number of revokes

Everytime we give a delegation to another node we count this as one 
delegation.
If the same record is delegated to several nodes we count one for each node.

Everytime a record has all its delegations revoked we count this as one 
revoke.

commit 9bde066f6eb46124168e5686fc41a323e67401e8
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Fri Mar 2 14:12:37 2012 +1100

READONLY: readonly fetch collapse. Make sure we only keep one single 
readonly fetch for a record in flight at a time.

---

Summary of changes:
 include/ctdb_private.h|4 +
 include/ctdb_protocol.h   |2 +
 server/ctdb_call.c|2 +
 server/ctdb_daemon.c  |  209 +
 server/ctdb_ltdb_server.c |   10 ++
 tools/ctdb.c  |2 +
 6 files changed, 229 insertions(+), 0 deletions(-)


Changeset truncated at 500 lines:

diff --git a/include/ctdb_private.h b/include/ctdb_private.h
index 8180722..86b664b 100644
--- a/include/ctdb_private.h
+++ b/include/ctdb_private.h
@@ -533,6 +533,10 @@ struct ctdb_db_context {
  struct ctdb_ltdb_header *header,
  TDB_DATA data);
 
+   /* used to track which records we are currently fetching with readonly
+  requests so we can avoid sending duplicates
+   */
+   struct trbt_tree *deferred_ro_fetch;
 };
 
 
diff --git a/include/ctdb_protocol.h b/include/ctdb_protocol.h
index efcc2cf..c874148 100644
--- a/include/ctdb_protocol.h
+++ b/include/ctdb_protocol.h
@@ -634,6 +634,8 @@ struct ctdb_statistics {
uint32_t num_recoveries;
struct timeval statistics_start_time;
struct timeval statistics_current_time;
+   uint32_t total_ro_delegations;
+   uint32_t total_ro_revokes;
 };
 
 /*
diff --git a/server/ctdb_call.c b/server/ctdb_call.c
index 2657f8b..1ece85a 100644
--- a/server/ctdb_call.c
+++ b/server/ctdb_call.c
@@ -513,6 +513,7 @@ void ctdb_request_call(struct ctdb_context *ctdb, struct 
ctdb_req_header *hdr)
 
if (header.flags  CTDB_REC_RO_REVOKE_COMPLETE) {
header.flags = 
~(CTDB_REC_RO_HAVE_DELEGATIONS|CTDB_REC_RO_HAVE_READONLY|CTDB_REC_RO_REVOKING_READONLY|CTDB_REC_RO_REVOKE_COMPLETE);
+   CTDB_INCREMENT_STAT(ctdb, total_ro_revokes);
if (ctdb_ltdb_store(ctdb_db, call-key, header, data) != 0) {
ctdb_fatal(ctdb, Failed to write header with cleared 
REVOKE flag);
}
@@ -619,6 +620,7 @@ void ctdb_request_call(struct ctdb_context *ctdb, struct 
ctdb_req_header *hdr)
}
 
ctdb_queue_packet(ctdb, r-hdr);
+   CTDB_INCREMENT_STAT(ctdb, total_ro_delegations);
 
talloc_free(r);
return;
diff --git a/server/ctdb_daemon.c b/server/ctdb_daemon.c
index c4f46b1..b09aaf3 100644
--- a/server/ctdb_daemon.c
+++ b/server/ctdb_daemon.c
@@ -27,6 +27,7 @@
 #include system/wait.h
 #include ../include/ctdb_client.h
 #include ../include/ctdb_private.h
+#include ../common/rb_tree.h
 #include sys/socket.h
 
 struct ctdb_client_pid_list {
@@ -384,6 +385,192 @@ static void daemon_incoming_packet_wrap(void *p, struct 
ctdb_req_header *hdr)
 }
 
 
+struct ctdb_deferred_fetch_call {
+   struct ctdb_deferred_fetch_call *next, *prev;
+   struct ctdb_req_call *c;
+   struct ctdb_daemon_packet_wrap *w;
+};
+
+struct ctdb_deferred_fetch_queue {
+   struct ctdb_deferred_fetch_call *deferred_calls;
+};
+
+struct ctdb_deferred_requeue {
+   struct ctdb_deferred_fetch_call *dfc;
+   struct ctdb_client *client;
+};
+
+
+/* called from a timer event and starts reprocessing the deferred call.*/
+static void reprocess_deferred_call(struct event_context *ev, struct 
timed_event *te, 
+  struct timeval t, void *private_data)
+{
+   struct ctdb_deferred_requeue *dfr = (struct ctdb_deferred_requeue 
*)private_data;
+   struct ctdb_client *client = dfr-client;
+
+   talloc_steal(client, dfr-dfc-c);
+   daemon_incoming_packet(client, (struct ctdb_req_header *)dfr-dfc-c);
+   talloc_free(dfr);
+}
+
+/* the referral context is destroyed either after a timeout or when the initial
+   fetch-lock has finished.
+   at this stage, immediately start reprocessing the queued up deferred

[SCM] CTDB repository - branch master updated - ctdb-1.12-225-g05e75de

2012-02-29 Thread Ronnie Sahlberg
The branch, master has been updated
   via  05e75dea9eaa0decc2909e0c2cf1715c81374200 (commit)
  from  7417d994c2a159f71d27d4bcd2f53684862eece3 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=master


- Log -
commit 05e75dea9eaa0decc2909e0c2cf1715c81374200
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Thu Mar 1 12:36:03 2012 +1100

New version 1.13

This will be the new long term stable branch for ctdb

Main new features are

read-only records: performance/scaling improvement for concurrent access to 
the same record from multiple nodes.

fetch-lock-collapse: performance/scaling improvement for concurrent write 
access to the same record from multiple nodes.

---

Summary of changes:
 packaging/RPM/ctdb.spec.in |4 
 1 files changed, 4 insertions(+), 0 deletions(-)


Changeset truncated at 500 lines:

diff --git a/packaging/RPM/ctdb.spec.in b/packaging/RPM/ctdb.spec.in
index 969a01b..1d6f570 100644
--- a/packaging/RPM/ctdb.spec.in
+++ b/packaging/RPM/ctdb.spec.in
@@ -146,6 +146,10 @@ development libraries for ctdb
 %{_libdir}/libctdb.a
 
 %changelog
+* Thu Mar 1 2012 : Version 1.13
+ - This is the new stable branch for modern features for ctdb.
+   Main new features are performance/scaling improvements for
+   concurrnet fetch and fetch_lock operations.
 * Tue Nov 8 2011 : Version 1.12
  - Add new tunable : AllowClientDBAttach that can be used to stop
client db access during maintenance operations


-- 
CTDB repository


[SCM] CTDB repository - annotated tag ctdb-1.13 created - ctdb-1.13

2012-02-29 Thread Ronnie Sahlberg
The annotated tag, ctdb-1.13 has been created
at  2c3cc0cd57e388eb48d780b691156ac074045570 (tag)
   tagging  05e75dea9eaa0decc2909e0c2cf1715c81374200 (commit)
  replaces  ctdb-1.12
 tagged by  Ronnie Sahlberg
on  Thu Mar 1 12:55:51 2012 +1100

- Log -
tag for 1.13
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)

iQEcBAABAgAGBQJPTtcyAAoJEB6eS+vwPaeGxCgIAJDHvkwLwF4cbbPS1dW3iUOa
izcIa8xCQBRQ2dUWodisynp17w9x6DMj1XIJmOPcRjWB4jrwUNK/KdXM3STq6E0R
5msHGP/s4tJ+QAV/92I855lWruurmd7lGxD48vG+CZEjYXfXUHd678McV8xWaAZW
ZmOlrZKmuYtzyxajBkrAL2qMeWGiiUgqrCDoK6Z8iSVusQTvvvXz4hjk44NyAeQR
VXi/kXLILaD2xT8AquNRuDxIobDN1gZEdwkL5QlGWpDNdQ82mNm4dnwvRRY57sD/
JEPSR0xQuLFPmj9Jg665syWQDwrAMRX1nLBf8NNr2DbwspVy0bmNWD3DjiDfsGo=
=2JQc
-END PGP SIGNATURE-

Andreas Schneider (1):
  libutil: Remove obsolete signal type cast.

Martin Schwenke (57):
  Clean up warnings: ‘tevent_loop_allow_nesting’ is deprecated
  Clean up warnings: remove changed_flags in monitor_helper
  Fix typo in ctdb_ltdb_store_server()
  Clean up warnings: log some unchecked return codes from function calls
  Clean up warnings: remove set but unused variable from rb_test
  Clean up warnings: remove unused variable alen from scsi_io
  Clean up warnings: remove unused function dump_packet()
  Clean up warnings: -Wunused-but-set-variable in tevent_signal.c
  Clean up warnings: add -Wno-format-zero-length to CFLAGS
  Fix ctdb scriptstatus -n all
  Clean up warnings: popt sure does some strange things
  Fix lib/replace
  Clean up warnings: rpcgen output contains unused variable.
  Added some #ifndefs to stop files being included multiple times.
  Move some common functions to common/ctdb_ltdb.c
  Rename ctdb_control_destructor() to ctdb_client_control_destructor()
  Rename ctdb_ctrl_getscriptstatus() parameter to avoid shadowing a global
  Rename ctdb_control_get_stat_history() local variable to avoid shadowing.
  Tests: change ctdb_takeover_tests.c to include ctdbd code
  Make some ctdb_takeover.c functions static
  Relax an out of memory message.
  Eventscripts: Make 40.fs_use use less processes and arguably clearer.
  Eventscripts - remove $0 from error messages in 40.fs_use
  Eventscripts - add facility to 10.interface to delete unmanaged IPs
  ctdb tool - simplify main() by taking most code out of a loop
  ctdb tool - commands that don't use the daemon can't take -n/--node
  ctdb tool - short circuit most of the logic in main for non-daemon 
commands
  ctdb tool - replace fprintf with DEBUG
  ctdb tool - move parsing of nodestring to where it is needed
  ctdb tool - generalise nodestring parsing for -n
  ctdb tool - factor out status printing functions
  ctdb tool - new command ctdb nodestatus [nodestring]
  ctdb tool - remove verify_node() function
  Rename struct traverse_state to allow tools C files to be #included.
  Tests - test code for parse_nodestring() function
  Tests - make a comment more accurate
  Tests: eventscripts - add tests for CTDB_DELETE_UNEXPECTED_IPS=yes
  ctdb tool - Fix parse_nodestring() related issues
  Tests - update test code for ctdb tool code and libctdb stubs
  libctdb - add ctdb_getvnnmap()
  ctdb tool - convert control_status() over to use just libctdb()
  Tests - more libctdb stubs and supporting state faking functions
  Tests - more libctdb stub test infrastructure
  Tests - allow tools/ctdb.c:main() to be used
  ctdb tool - free some memory used by popt
  Tests - add a version of the ctdb tool that compiles against libctdb stubs
  Tests: libctdb stubs should fail when current node disconnected
  Tests: fix usage message in ctdb_tool_libctdb
  Tests: initial ctdb tool testcases
  Tests - let run_tests.sh take a list of tests to run
  Tests - ctdb tool - fix ctdb status test
  Tests - ctdb tool - add verbose option and output filtering
  Tests: new test to check that ctdb delip removes the IP from interface.
  Tests: eventscripts - make init test pass with recent change to 
10.interface
  Tests: eventscripts - fix breakage in some NFS tests
  Tests - IP allocation simulation - LCP2 = non-deterministic
  Test - IP allocation simulation - add -e option to run the daemon's 
algorithm

Mathieu Parent (11):
  config/functions: CTDB_VARDIR is /var/lib/ctdb on Debian-like systems
  config can be in /etc/default/ instead of /etc/sysconfig/
  config can be in /etc/default/ instead of /etc/sysconfig/ (ctdb_gnostics 
part)
  Nagios plugin for CTDB
  Fix ctdb-crash-cleanup sysconfig handling
  Remove zero-length gnu_printf format string in ctdb_daemon.c (gcc warning)
  Move platform-specific code to common/system_*
  GNU/Hurd support
  Add kFreeBSD support
  Define ETIME

[SCM] CTDB repository - branch 1.2.39 updated - ctdb-1.9.1-499-gf053a6d

2012-02-28 Thread Ronnie Sahlberg
The branch, 1.2.39 has been updated
   via  f053a6d2948a1933d38d6cdd4cae55349e71b7d4 (commit)
  from  6f6dac21f93c38c3abcbebc1b786b4da2ef9f563 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=1.2.39


- Log -
commit f053a6d2948a1933d38d6cdd4cae55349e71b7d4
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Wed Feb 29 12:25:41 2012 +1100

Vacuuming: change default timeout to 120 seconds

S1035431

---

Summary of changes:
 server/ctdb_tunables.c |2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)


Changeset truncated at 500 lines:

diff --git a/server/ctdb_tunables.c b/server/ctdb_tunables.c
index eb90f51..62a4dd4 100644
--- a/server/ctdb_tunables.c
+++ b/server/ctdb_tunables.c
@@ -58,7 +58,7 @@ static const struct {
{ RecoveryDropAllIPs, 120,  offsetof(struct ctdb_tunable, 
recovery_drop_all_ips) },
{ VerifyRecoveryLock,   1,  offsetof(struct ctdb_tunable, 
verify_recovery_lock) },
{ VacuumDefaultInterval, 10,  offsetof(struct ctdb_tunable, 
vacuum_default_interval) },
-   { VacuumMaxRunTime, 30,  offsetof(struct ctdb_tunable, 
vacuum_max_run_time) },
+   { VacuumMaxRunTime, 120,  offsetof(struct ctdb_tunable, 
vacuum_max_run_time) },
{ RepackLimit,  1,  offsetof(struct ctdb_tunable, 
repack_limit) },
{ VacuumLimit,   5000,  offsetof(struct ctdb_tunable, 
vacuum_limit) },
{ VacuumMinInterval,   10,  offsetof(struct ctdb_tunable, 
vacuum_min_interval) },


-- 
CTDB repository


[SCM] CTDB repository - branch 1.2.40 updated - ctdb-1.9.1-542-gfd33e6f

2012-02-28 Thread Ronnie Sahlberg
The branch, 1.2.40 has been updated
   via  fd33e6ff1e349e3d6d1d2e78ab14942c97aba731 (commit)
  from  a02fa85678cc5061042ab6d448b8a3f5993f2d70 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=1.2.40


- Log -
commit fd33e6ff1e349e3d6d1d2e78ab14942c97aba731
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Wed Feb 29 12:25:41 2012 +1100

Vacuuming: change default timeout to 120 seconds

S1035431

---

Summary of changes:
 server/ctdb_tunables.c |2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)


Changeset truncated at 500 lines:

diff --git a/server/ctdb_tunables.c b/server/ctdb_tunables.c
index eb90f51..62a4dd4 100644
--- a/server/ctdb_tunables.c
+++ b/server/ctdb_tunables.c
@@ -58,7 +58,7 @@ static const struct {
{ RecoveryDropAllIPs, 120,  offsetof(struct ctdb_tunable, 
recovery_drop_all_ips) },
{ VerifyRecoveryLock,   1,  offsetof(struct ctdb_tunable, 
verify_recovery_lock) },
{ VacuumDefaultInterval, 10,  offsetof(struct ctdb_tunable, 
vacuum_default_interval) },
-   { VacuumMaxRunTime, 30,  offsetof(struct ctdb_tunable, 
vacuum_max_run_time) },
+   { VacuumMaxRunTime, 120,  offsetof(struct ctdb_tunable, 
vacuum_max_run_time) },
{ RepackLimit,  1,  offsetof(struct ctdb_tunable, 
repack_limit) },
{ VacuumLimit,   5000,  offsetof(struct ctdb_tunable, 
vacuum_limit) },
{ VacuumMinInterval,   10,  offsetof(struct ctdb_tunable, 
vacuum_min_interval) },


-- 
CTDB repository


[SCM] CTDB repository - branch master updated - ctdb-1.12-223-g5ae94c6

2012-02-28 Thread Ronnie Sahlberg
The branch, master has been updated
   via  5ae94c6b9b3000a6c79fccaaea1e007ebd5be1a9 (commit)
  from  49791db7dc74cffd7e88bd73091590cdc1909328 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=master


- Log -
commit 5ae94c6b9b3000a6c79fccaaea1e007ebd5be1a9
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Wed Feb 29 12:29:22 2012 +1100

Vacuuming: change default timeout to 120 seconds

---

Summary of changes:
 server/ctdb_tunables.c |2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)


Changeset truncated at 500 lines:

diff --git a/server/ctdb_tunables.c b/server/ctdb_tunables.c
index c5e15c1..76af85d 100644
--- a/server/ctdb_tunables.c
+++ b/server/ctdb_tunables.c
@@ -60,7 +60,7 @@ static const struct {
{ VerifyRecoveryLock,   1,  offsetof(struct ctdb_tunable, 
verify_recovery_lock), false },
{ VacuumInterval,   10,  offsetof(struct ctdb_tunable, 
vacuum_interval), false },
{ VacuumDefaultInterval, 10,  offsetof(struct ctdb_tunable, 
vacuum_default_interval), true },
-   { VacuumMaxRunTime, 30,  offsetof(struct ctdb_tunable, 
vacuum_max_run_time), false },
+   { VacuumMaxRunTime, 120,  offsetof(struct ctdb_tunable, 
vacuum_max_run_time), false },
{ RepackLimit,  1,  offsetof(struct ctdb_tunable, 
repack_limit), false },
{ VacuumLimit,   5000,  offsetof(struct ctdb_tunable, 
vacuum_limit), false },
{ VacuumMinInterval,   10,  offsetof(struct ctdb_tunable, 
vacuum_min_interval), true },


-- 
CTDB repository


[SCM] CTDB repository - branch master updated - ctdb-1.12-224-g7417d99

2012-02-28 Thread Ronnie Sahlberg
The branch, master has been updated
   via  7417d994c2a159f71d27d4bcd2f53684862eece3 (commit)
  from  5ae94c6b9b3000a6c79fccaaea1e007ebd5be1a9 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=master


- Log -
commit 7417d994c2a159f71d27d4bcd2f53684862eece3
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Wed Feb 29 16:09:24 2012 +1100

READONLY: skip vacuuming or deleting records with readonly delegations.
they are hot. wait until they have been revoked before we recall them.

---

Summary of changes:
 server/ctdb_recover.c |   14 ++
 server/ctdb_vacuum.c  |6 ++
 2 files changed, 20 insertions(+), 0 deletions(-)


Changeset truncated at 500 lines:

diff --git a/server/ctdb_recover.c b/server/ctdb_recover.c
index 3d56f77..06b5ed1 100644
--- a/server/ctdb_recover.c
+++ b/server/ctdb_recover.c
@@ -954,6 +954,20 @@ static int delete_tdb_record(struct ctdb_context *ctdb, 
struct ctdb_db_context *
return -1;  
}
 
+   /* do not allow deleting record that have readonly flags set. */
+   if (hdr-flags  
(CTDB_REC_RO_HAVE_DELEGATIONS|CTDB_REC_RO_HAVE_READONLY|CTDB_REC_RO_REVOKING_READONLY|CTDB_REC_RO_REVOKE_COMPLETE))
 {
+   tdb_chainunlock(ctdb_db-ltdb-tdb, key);
+   DEBUG(DEBUG_INFO,(__location__  Skipping record with readonly 
flags set\n));
+   free(data.dptr);
+   return -1;  
+   }
+   if (hdr2-flags  
(CTDB_REC_RO_HAVE_DELEGATIONS|CTDB_REC_RO_HAVE_READONLY|CTDB_REC_RO_REVOKING_READONLY|CTDB_REC_RO_REVOKE_COMPLETE))
 {
+   tdb_chainunlock(ctdb_db-ltdb-tdb, key);
+   DEBUG(DEBUG_INFO,(__location__  Skipping record with readonly 
flags set\n));
+   free(data.dptr);
+   return -1;  
+   }
+
if (hdr2-dmaster == ctdb-pnn) {
tdb_chainunlock(ctdb_db-ltdb-tdb, key);
DEBUG(DEBUG_INFO,(__location__  Attempted delete record where 
we are the dmaster\n));
diff --git a/server/ctdb_vacuum.c b/server/ctdb_vacuum.c
index e0e1e3b..b492f9b 100644
--- a/server/ctdb_vacuum.c
+++ b/server/ctdb_vacuum.c
@@ -491,6 +491,12 @@ static int delete_record_traverse(void *param, void *data)
 
header = (struct ctdb_ltdb_header *)tdb_data.dptr;
 
+   if (header-flags  
(CTDB_REC_RO_HAVE_DELEGATIONS|CTDB_REC_RO_HAVE_READONLY|CTDB_REC_RO_REVOKING_READONLY|CTDB_REC_RO_REVOKE_COMPLETE))
 {
+ /* The record has readonly flags set. skip deleting */
+   vdata-delete_skipped++;
+   goto done;
+   }
+
if (header-dmaster != ctdb-pnn) {
/* The record has been migrated off the node. Skip. */
vdata-delete_skipped++;


-- 
CTDB repository


[SCM] CTDB repository - branch 1.2.39 updated - ctdb-1.9.1-498-g6f6dac2

2012-02-27 Thread Ronnie Sahlberg
The branch, 1.2.39 has been updated
   via  6f6dac21f93c38c3abcbebc1b786b4da2ef9f563 (commit)
   via  e86e0e8547593eb8ea2d5c65892de81eac694783 (commit)
   via  abd73b5494906c915c3379852524495957a1bf50 (commit)
  from  1260c1b9f301dccd497be8b818be5463858f0603 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=1.2.39


- Log -
commit 6f6dac21f93c38c3abcbebc1b786b4da2ef9f563
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Tue Feb 28 08:44:06 2012 +1100

new version 1.2.39-3

commit e86e0e8547593eb8ea2d5c65892de81eac694783
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Tue Feb 28 06:44:08 2012 +1100

Add a tunable variable to control how long we defer after a ctdb addip 
until we force a rebalance and try to failback addresses onto this node

commit abd73b5494906c915c3379852524495957a1bf50
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Mon Feb 27 19:16:08 2012 +1100

When adding ips to nodes, set up a deferred rebalance for the whole node to 
trigger after 60 seconds in case the normal ipreallocated is not sufficient to 
trigger rebalance.

S1035306

---

Summary of changes:
 include/ctdb_private.h |2 +
 include/ctdb_protocol.h|5 
 packaging/RPM/ctdb.spec.in |5 +++-
 server/ctdb_recoverd.c |   56 
 server/ctdb_takeover.c |   40 +++
 server/ctdb_tunables.c |3 +-
 tools/ctdb.c   |   45 +--
 7 files changed, 151 insertions(+), 5 deletions(-)


Changeset truncated at 500 lines:

diff --git a/include/ctdb_private.h b/include/ctdb_private.h
index e3a9a15..b490b4d 100644
--- a/include/ctdb_private.h
+++ b/include/ctdb_private.h
@@ -122,6 +122,7 @@ struct ctdb_tunable {
uint32_t vacuum_fast_path_count;
uint32_t lcp2_public_ip_assignment;
uint32_t allow_client_db_attach;
+   uint32_t deferred_rebalance_on_node_add;
 };
 
 /*
@@ -1442,6 +1443,7 @@ bool lcp2_failback(struct ctdb_context *ctdb,
   struct ctdb_public_ip_list *all_ips,
   uint32_t *lcp2_imbalances,
   bool *newly_healthy);
+void lcp2_forcerebalance(struct ctdb_context *ctdb, uint32_t pnn);
 void ctdb_takeover_run_core(struct ctdb_context *ctdb,
struct ctdb_node_map *nodemap,
struct ctdb_public_ip_list **all_ips_p);
diff --git a/include/ctdb_protocol.h b/include/ctdb_protocol.h
index 0422afe..1a116c1 100644
--- a/include/ctdb_protocol.h
+++ b/include/ctdb_protocol.h
@@ -127,6 +127,11 @@ struct ctdb_call_info {
  */
 #define CTDB_SRVID_TAKEOVER_RUN 0xFB00LL
 
+/* request recovery daemon to rebalance ips for a node.
+   input is uint32_t for the node id.
+*/
+#define CTDB_SRVID_REBALANCE_NODE 0xFB01LL
+
 /* A message id to ask the recovery daemon to temporarily disable the
public ip checks
 */
diff --git a/packaging/RPM/ctdb.spec.in b/packaging/RPM/ctdb.spec.in
index 7dd61bf..4f46827 100644
--- a/packaging/RPM/ctdb.spec.in
+++ b/packaging/RPM/ctdb.spec.in
@@ -4,7 +4,7 @@ Summary: Clustered TDB
 Vendor: Samba Team
 Packager: Samba Team sa...@samba.org
 Version: 1.2.39
-Release: 2GITHASH
+Release: 3GITHASH
 Epoch: 0
 License: GNU GPL version 3
 Group: System Environment/Daemons
@@ -144,6 +144,9 @@ development libraries for ctdb
 %{_libdir}/libctdb.a
 
 %changelog
+* Tue Feb 28 2012 : version 1.2.39-3
+ - fix for memory corruption S1034147
+ - fix for imbalance after addip S1035305
 * Tue Feb 14 2012 : Version 1.2.39-2
  - Fix for ip reallocation bug resulting in poor istribution of addresses
 * Thu Nov 17 2011 : Version 1.2.39
diff --git a/server/ctdb_recoverd.c b/server/ctdb_recoverd.c
index 4963c3f..d206d3d 100644
--- a/server/ctdb_recoverd.c
+++ b/server/ctdb_recoverd.c
@@ -65,6 +65,7 @@ struct ctdb_recoverd {
struct ip_reallocate_list *reallocate_callers;
TALLOC_CTX *ip_check_disable_ctx;
struct ctdb_control_get_ifaces *ifaces;
+   TALLOC_CTX *deferred_rebalance_ctx;
 };
 
 #define CONTROL_TIMEOUT() timeval_current_ofs(ctdb-tunable.recover_timeout, 0)
@@ -1941,6 +1942,57 @@ static void reenable_ip_check(struct event_context *ev, 
struct timed_event *te,
 }
 
 
+static void ctdb_rebalance_timeout(struct event_context *ev, struct 
timed_event *te, 
+ struct timeval t, void *p)
+{
+   struct ctdb_recoverd *rec = talloc_get_type(p, struct ctdb_recoverd);
+   struct ctdb_context *ctdb = rec-ctdb;
+   int ret;
+
+   DEBUG(DEBUG_NOTICE,(Rebalance all nodes that have had ip assignment 
changes.\n));
+
+   ret = ctdb_takeover_run(ctdb, rec-nodemap);
+   if (ret != 0) {
+   DEBUG(DEBUG_ERR, (__location__  Unable to setup public 
takeover addresses

[SCM] CTDB repository - branch 1.2.40 updated - ctdb-1.9.1-541-ga02fa85

2012-02-27 Thread Ronnie Sahlberg
The branch, 1.2.40 has been updated
   via  a02fa85678cc5061042ab6d448b8a3f5993f2d70 (commit)
   via  710b0c7b949cc4f93f64c3a02c24fbda1f71 (commit)
  from  7bbd5f6c68f0a816249eddbbc64f34bd929c43d3 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=1.2.40


- Log -
commit a02fa85678cc5061042ab6d448b8a3f5993f2d70
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Tue Feb 28 06:44:08 2012 +1100

Add a tunable variable to control how long we defer after a ctdb addip 
until we force a rebalance and try to failback addresses onto this node

commit 710b0c7b949cc4f93f64c3a02c24fbda1f71
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Mon Feb 27 19:16:08 2012 +1100

When adding ips to nodes, set up a deferred rebalance for the whole node to 
trigger after 60 seconds in case the normal ipreallocated is not sufficient to 
trigger rebalance.

S1035306

---

Summary of changes:
 include/ctdb_private.h  |2 +
 include/ctdb_protocol.h |5 
 server/ctdb_recoverd.c  |   56 +++
 server/ctdb_takeover.c  |   40 +
 server/ctdb_tunables.c  |3 +-
 tools/ctdb.c|   45 +++--
 6 files changed, 147 insertions(+), 4 deletions(-)


Changeset truncated at 500 lines:

diff --git a/include/ctdb_private.h b/include/ctdb_private.h
index 4edb647..7d0a6d8 100644
--- a/include/ctdb_private.h
+++ b/include/ctdb_private.h
@@ -122,6 +122,7 @@ struct ctdb_tunable {
uint32_t vacuum_fast_path_count;
uint32_t lcp2_public_ip_assignment;
uint32_t allow_client_db_attach;
+   uint32_t deferred_rebalance_on_node_add;
 };
 
 /*
@@ -1448,6 +1449,7 @@ bool lcp2_failback(struct ctdb_context *ctdb,
   struct ctdb_public_ip_list *all_ips,
   uint32_t *lcp2_imbalances,
   bool *newly_healthy);
+void lcp2_forcerebalance(struct ctdb_context *ctdb, uint32_t pnn);
 void ctdb_takeover_run_core(struct ctdb_context *ctdb,
struct ctdb_node_map *nodemap,
struct ctdb_public_ip_list **all_ips_p);
diff --git a/include/ctdb_protocol.h b/include/ctdb_protocol.h
index f4019ab..efcc2cf 100644
--- a/include/ctdb_protocol.h
+++ b/include/ctdb_protocol.h
@@ -130,6 +130,11 @@ struct ctdb_call_info {
  */
 #define CTDB_SRVID_TAKEOVER_RUN 0xFB00LL
 
+/* request recovery daemon to rebalance ips for a node.
+   input is uint32_t for the node id.
+*/
+#define CTDB_SRVID_REBALANCE_NODE 0xFB01LL
+
 /* A message id to ask the recovery daemon to temporarily disable the
public ip checks
 */
diff --git a/server/ctdb_recoverd.c b/server/ctdb_recoverd.c
index ce2aad8..bf19089 100644
--- a/server/ctdb_recoverd.c
+++ b/server/ctdb_recoverd.c
@@ -65,6 +65,7 @@ struct ctdb_recoverd {
struct ip_reallocate_list *reallocate_callers;
TALLOC_CTX *ip_check_disable_ctx;
struct ctdb_control_get_ifaces *ifaces;
+   TALLOC_CTX *deferred_rebalance_ctx;
 };
 
 #define CONTROL_TIMEOUT() timeval_current_ofs(ctdb-tunable.recover_timeout, 0)
@@ -1942,6 +1943,57 @@ static void reenable_ip_check(struct event_context *ev, 
struct timed_event *te,
 }
 
 
+static void ctdb_rebalance_timeout(struct event_context *ev, struct 
timed_event *te, 
+ struct timeval t, void *p)
+{
+   struct ctdb_recoverd *rec = talloc_get_type(p, struct ctdb_recoverd);
+   struct ctdb_context *ctdb = rec-ctdb;
+   int ret;
+
+   DEBUG(DEBUG_NOTICE,(Rebalance all nodes that have had ip assignment 
changes.\n));
+
+   ret = ctdb_takeover_run(ctdb, rec-nodemap);
+   if (ret != 0) {
+   DEBUG(DEBUG_ERR, (__location__  Unable to setup public 
takeover addresses. ctdb_takeover_run() failed.\n));
+   rec-need_takeover_run = true;
+   }
+
+   talloc_free(rec-deferred_rebalance_ctx);
+   rec-deferred_rebalance_ctx = NULL;
+}
+
+   
+static void recd_node_rebalance_handler(struct ctdb_context *ctdb, uint64_t 
srvid, 
+TDB_DATA data, void *private_data)
+{
+   uint32_t pnn;
+   struct ctdb_recoverd *rec = talloc_get_type(private_data, struct 
ctdb_recoverd);
+
+   if (data.dsize != sizeof(uint32_t)) {
+   DEBUG(DEBUG_ERR,(__location__  Incorrect size of node 
rebalance message. Was %zd but expected %zd bytes\n, data.dsize, 
sizeof(uint32_t)));
+   return;
+   }
+
+   if (ctdb-tunable.deferred_rebalance_on_node_add == 0) {
+   return;
+   }
+
+   pnn = *(uint32_t *)data.dptr[0];
+
+   lcp2_forcerebalance(ctdb, pnn);
+   DEBUG(DEBUG_NOTICE,(Received message to perform node rebalancing for 
node %d\n, pnn));
+
+   if (rec-deferred_rebalance_ctx != NULL

[SCM] CTDB repository - branch master updated - ctdb-1.12-222-g49791db

2012-02-27 Thread Ronnie Sahlberg
The branch, master has been updated
   via  49791db7dc74cffd7e88bd73091590cdc1909328 (commit)
   via  4340263b219d75c39f8de22abe3f6f1c1ee63ea2 (commit)
  from  02b62482164a3c69715949074feb7f191a29d534 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=master


- Log -
commit 49791db7dc74cffd7e88bd73091590cdc1909328
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Tue Feb 28 06:58:59 2012 +1100

Add a tunable variable to control how long we defer after a ctdb addip 
until we force a rebalance and try to failback addresses onto this node
Have it default to 300 seconds.

commit 4340263b219d75c39f8de22abe3f6f1c1ee63ea2
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Tue Feb 28 06:56:04 2012 +1100

When adding ips to nodes, set up a deferred rebalance for the whole node to 
trigger after 60 seconds in case the normal ipreallocated is not sufficient to 
trigger rebalance.

---

Summary of changes:
 include/ctdb_private.h  |2 +
 include/ctdb_protocol.h |5 
 server/ctdb_recoverd.c  |   56 +++
 server/ctdb_takeover.c  |   40 +
 server/ctdb_tunables.c  |3 +-
 tools/ctdb.c|   52 ++-
 6 files changed, 146 insertions(+), 12 deletions(-)


Changeset truncated at 500 lines:

diff --git a/include/ctdb_private.h b/include/ctdb_private.h
index 4b7e675..3039953 100644
--- a/include/ctdb_private.h
+++ b/include/ctdb_private.h
@@ -124,6 +124,7 @@ struct ctdb_tunable {
uint32_t lcp2_public_ip_assignment;
uint32_t allow_client_db_attach;
uint32_t recover_pdb_by_seqnum;
+   uint32_t deferred_rebalance_on_node_add;
 };
 
 /*
@@ -1132,6 +1133,7 @@ int ctdb_set_single_public_ip(struct ctdb_context *ctdb,
 int ctdb_set_event_script(struct ctdb_context *ctdb, const char *script);
 int ctdb_set_event_script_dir(struct ctdb_context *ctdb, const char 
*script_dir);
 int ctdb_set_notification_script(struct ctdb_context *ctdb, const char 
*script);
+void lcp2_forcerebalance(struct ctdb_context *ctdb, uint32_t pnn);
 int ctdb_takeover_run(struct ctdb_context *ctdb, struct ctdb_node_map 
*nodemap);
 
 int32_t ctdb_control_tcp_client(struct ctdb_context *ctdb, uint32_t client_id, 
diff --git a/include/ctdb_protocol.h b/include/ctdb_protocol.h
index 4cf2789..3e466d8 100644
--- a/include/ctdb_protocol.h
+++ b/include/ctdb_protocol.h
@@ -130,6 +130,11 @@ struct ctdb_call_info {
  */
 #define CTDB_SRVID_TAKEOVER_RUN 0xFB00LL
 
+/* request recovery daemon to rebalance ips for a node.
+   input is uint32_t for the node id.
+*/
+#define CTDB_SRVID_REBALANCE_NODE 0xFB01LL
+
 /* A message id to ask the recovery daemon to temporarily disable the
public ip checks
 */
diff --git a/server/ctdb_recoverd.c b/server/ctdb_recoverd.c
index b5a6d63..642679b 100644
--- a/server/ctdb_recoverd.c
+++ b/server/ctdb_recoverd.c
@@ -65,6 +65,7 @@ struct ctdb_recoverd {
struct ip_reallocate_list *reallocate_callers;
TALLOC_CTX *ip_check_disable_ctx;
struct ctdb_control_get_ifaces *ifaces;
+   TALLOC_CTX *deferred_rebalance_ctx;
 };
 
 #define CONTROL_TIMEOUT() timeval_current_ofs(ctdb-tunable.recover_timeout, 0)
@@ -2068,6 +2069,57 @@ static void reenable_ip_check(struct event_context *ev, 
struct timed_event *te,
 }
 
 
+static void ctdb_rebalance_timeout(struct event_context *ev, struct 
timed_event *te, 
+ struct timeval t, void *p)
+{
+   struct ctdb_recoverd *rec = talloc_get_type(p, struct ctdb_recoverd);
+   struct ctdb_context *ctdb = rec-ctdb;
+   int ret;
+
+   DEBUG(DEBUG_NOTICE,(Rebalance all nodes that have had ip assignment 
changes.\n));
+
+   ret = ctdb_takeover_run(ctdb, rec-nodemap);
+   if (ret != 0) {
+   DEBUG(DEBUG_ERR, (__location__  Unable to setup public 
takeover addresses. ctdb_takeover_run() failed.\n));
+   rec-need_takeover_run = true;
+   }
+
+   talloc_free(rec-deferred_rebalance_ctx);
+   rec-deferred_rebalance_ctx = NULL;
+}
+
+   
+static void recd_node_rebalance_handler(struct ctdb_context *ctdb, uint64_t 
srvid, 
+TDB_DATA data, void *private_data)
+{
+   uint32_t pnn;
+   struct ctdb_recoverd *rec = talloc_get_type(private_data, struct 
ctdb_recoverd);
+
+   if (data.dsize != sizeof(uint32_t)) {
+   DEBUG(DEBUG_ERR,(__location__  Incorrect size of node 
rebalance message. Was %zd but expected %zd bytes\n, data.dsize, 
sizeof(uint32_t)));
+   return;
+   }
+
+   if (ctdb-tunable.deferred_rebalance_on_node_add == 0) {
+   return;
+   }
+
+   pnn = *(uint32_t *)data.dptr[0];
+
+   lcp2_forcerebalance(ctdb, pnn);
+   DEBUG(DEBUG_NOTICE

[SCM] CTDB repository - branch 1.2.40 updated - ctdb-1.9.1-539-g7bbd5f6

2012-02-26 Thread Ronnie Sahlberg
The branch, 1.2.40 has been updated
   via  7bbd5f6c68f0a816249eddbbc64f34bd929c43d3 (commit)
   via  4b34a14912e8fd58f0ce755ef4378cce931b8828 (commit)
   via  f78136d04ff41e8d22c61351c73f1607cbda7dc5 (commit)
  from  22138474060cba8c3a9b8b863488655ff6926258 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=1.2.40


- Log -
commit 7bbd5f6c68f0a816249eddbbc64f34bd929c43d3
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Mon Feb 27 07:18:19 2012 +1100

Make KILLTCP structure a child of VNN so that it is freed at the same time
the referenced VNN structure is.

Also, remove the circular reference between the two objects KIPPCTP and VNN

commit 4b34a14912e8fd58f0ce755ef4378cce931b8828
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Wed Feb 22 17:43:04 2012 +1100

make tdb valgrind-happy

commit f78136d04ff41e8d22c61351c73f1607cbda7dc5
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Wed Feb 22 17:38:12 2012 +1100

Eventscripts: remove the horrible horrible circular reference between state 
and callback since these two structures do not even share the same parent 
talloc context.
Instead, tie them together via referencing a permanent linked list hung off 
the ctdb structure.

---

Summary of changes:
 include/ctdb_private.h |3 +++
 lib/tdb/common/hash.c  |1 +
 server/ctdb_takeover.c |   22 +++---
 server/eventscript.c   |   26 +++---
 4 files changed, 42 insertions(+), 10 deletions(-)


Changeset truncated at 500 lines:

diff --git a/include/ctdb_private.h b/include/ctdb_private.h
index b48f84a..4edb647 100644
--- a/include/ctdb_private.h
+++ b/include/ctdb_private.h
@@ -495,6 +495,9 @@ struct ctdb_context {
 
/* Used to defer db attach requests while in recovery mode */
struct ctdb_deferred_attach_context *deferred_attach;
+
+   /* list of event script callback functions that are active */
+   struct event_script_callback *script_callbacks;
 };
 
 struct ctdb_db_context {
diff --git a/lib/tdb/common/hash.c b/lib/tdb/common/hash.c
index c07297e..e8f7555 100644
--- a/lib/tdb/common/hash.c
+++ b/lib/tdb/common/hash.c
@@ -23,6 +23,7 @@
License along with this library; if not, see http://www.gnu.org/licenses/.
 */
 #include tdb_private.h
+#define VALGRIND
 
 /* This is based on the hash algorithm from gdbm */
 unsigned int tdb_old_hash(TDB_DATA *key)
diff --git a/server/ctdb_takeover.c b/server/ctdb_takeover.c
index b2e1a8d..9cbafaf 100644
--- a/server/ctdb_takeover.c
+++ b/server/ctdb_takeover.c
@@ -2880,9 +2880,25 @@ static void ctdb_tickle_sentenced_connections(struct 
event_context *ev, struct t
  */
 static int ctdb_killtcp_destructor(struct ctdb_kill_tcp *killtcp)
 {
-   if (killtcp-vnn) {
-   killtcp-vnn-killtcp = NULL;
+   struct ctdb_vnn *tmpvnn;
+
+   /* verify that this vnn is still active */
+   for (tmpvnn = killtcp-ctdb-vnn; tmpvnn; tmpvnn = tmpvnn-next) {
+   if (tmpvnn == killtcp-vnn) {
+   break;
+   }
+   }
+
+   if (tmpvnn == NULL) {
+   return 0;
}
+
+   if (killtcp-vnn-killtcp != killtcp) {
+   return 0;
+   }
+
+   killtcp-vnn-killtcp = NULL;
+
return 0;
 }
 
@@ -2937,7 +2953,7 @@ static int ctdb_killtcp_add_connection(struct 
ctdb_context *ctdb,
   a new structure
 */
if (killtcp == NULL) {
-   killtcp = talloc_zero(ctdb, struct ctdb_kill_tcp);
+   killtcp = talloc_zero(vnn, struct ctdb_kill_tcp);
CTDB_NO_MEMORY(ctdb, killtcp);
 
killtcp-vnn = vnn;
diff --git a/server/eventscript.c b/server/eventscript.c
index 17cc3d4..c42aaa4 100644
--- a/server/eventscript.c
+++ b/server/eventscript.c
@@ -26,6 +26,7 @@
 #include ../include/ctdb_private.h
 #include lib/tevent/tevent.h
 #include ../common/rb_tree.h
+#include lib/util/dlinklist.h
 
 static void ctdb_event_script_timeout(struct event_context *ev, struct 
timed_event *te, struct timeval t, void *p);
 
@@ -41,7 +42,8 @@ static void sigterm(int sig)
 
 /* This is attached to the event script state. */
 struct event_script_callback {
-   struct ctdb_event_script_state *state;
+   struct event_script_callback *next, *prev;
+   struct ctdb_context *ctdb;
 
/* Warning: this can free us! */
void (*fn)(struct ctdb_context *, int, void *);
@@ -611,8 +613,18 @@ static int event_script_destructor(struct 
ctdb_event_script_state *state)
}
 
/* This is allowed to free us; talloc will prevent double free anyway,
-* but beware if you call this outside the destructor! */
-   callback = state-callback;
+* but beware if you call this outside the destructor!
+* the callback hangs off a different

[SCM] CTDB repository - branch master updated - ctdb-1.12-220-g02b6248

2012-02-26 Thread Ronnie Sahlberg
The branch, master has been updated
   via  02b62482164a3c69715949074feb7f191a29d534 (commit)
   via  a95c02da6c67dc4bd8716b75318a4188301df6f9 (commit)
  from  d306c3c9a53e012c412c96ab9743de6cd96826e0 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=master


- Log -
commit 02b62482164a3c69715949074feb7f191a29d534
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Mon Feb 27 07:18:19 2012 +1100

Make KILLTCP structure a child of VNN so that it is freed at the same time
the referenced VNN structure is.

Also, remove the circular reference between the two objects KIPPCTP and VNN

commit a95c02da6c67dc4bd8716b75318a4188301df6f9
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Wed Feb 22 17:38:12 2012 +1100

Eventscripts: remove the horrible horrible circular reference between state 
and callback since these two structures do not even share the same parent 
talloc context.
Instead, tie them together via referencing a permanent linked list hung off 
the ctdb structure.

---

Summary of changes:
 include/ctdb_private.h |3 +++
 server/ctdb_takeover.c |   22 +++---
 server/eventscript.c   |   26 +++---
 3 files changed, 41 insertions(+), 10 deletions(-)


Changeset truncated at 500 lines:

diff --git a/include/ctdb_private.h b/include/ctdb_private.h
index 272b94a..4b7e675 100644
--- a/include/ctdb_private.h
+++ b/include/ctdb_private.h
@@ -486,6 +486,9 @@ struct ctdb_context {
 
/* if we are a child process, do we have a domain socket to send 
controls on */
bool can_send_controls;
+
+   /* list of event script callback functions that are active */
+   struct event_script_callback *script_callbacks;
 };
 
 struct ctdb_db_context {
diff --git a/server/ctdb_takeover.c b/server/ctdb_takeover.c
index 830b751..c91d2f7 100644
--- a/server/ctdb_takeover.c
+++ b/server/ctdb_takeover.c
@@ -2934,9 +2934,25 @@ static void ctdb_tickle_sentenced_connections(struct 
event_context *ev, struct t
  */
 static int ctdb_killtcp_destructor(struct ctdb_kill_tcp *killtcp)
 {
-   if (killtcp-vnn) {
-   killtcp-vnn-killtcp = NULL;
+   struct ctdb_vnn *tmpvnn;
+
+   /* verify that this vnn is still active */
+   for (tmpvnn = killtcp-ctdb-vnn; tmpvnn; tmpvnn = tmpvnn-next) {
+   if (tmpvnn == killtcp-vnn) {
+   break;
+   }
+   }
+
+   if (tmpvnn == NULL) {
+   return 0;
}
+
+   if (killtcp-vnn-killtcp != killtcp) {
+   return 0;
+   }
+
+   killtcp-vnn-killtcp = NULL;
+
return 0;
 }
 
@@ -2991,7 +3007,7 @@ static int ctdb_killtcp_add_connection(struct 
ctdb_context *ctdb,
   a new structure
 */
if (killtcp == NULL) {
-   killtcp = talloc_zero(ctdb, struct ctdb_kill_tcp);
+   killtcp = talloc_zero(vnn, struct ctdb_kill_tcp);
CTDB_NO_MEMORY(ctdb, killtcp);
 
killtcp-vnn = vnn;
diff --git a/server/eventscript.c b/server/eventscript.c
index 722ebec..146da9d 100644
--- a/server/eventscript.c
+++ b/server/eventscript.c
@@ -26,6 +26,7 @@
 #include ../include/ctdb_private.h
 #include lib/tevent/tevent.h
 #include ../common/rb_tree.h
+#include lib/util/dlinklist.h
 
 static void ctdb_event_script_timeout(struct event_context *ev, struct 
timed_event *te, struct timeval t, void *p);
 
@@ -41,7 +42,8 @@ static void sigterm(int sig)
 
 /* This is attached to the event script state. */
 struct event_script_callback {
-   struct ctdb_event_script_state *state;
+   struct event_script_callback *next, *prev;
+   struct ctdb_context *ctdb;
 
/* Warning: this can free us! */
void (*fn)(struct ctdb_context *, int, void *);
@@ -611,8 +613,18 @@ static int event_script_destructor(struct 
ctdb_event_script_state *state)
}
 
/* This is allowed to free us; talloc will prevent double free anyway,
-* but beware if you call this outside the destructor! */
-   callback = state-callback;
+* but beware if you call this outside the destructor!
+* the callback hangs off a different context so we walk the list
+* of active callbacks until we find the one state points to.
+* if we cant find it it means the callback has been removed.
+*/
+   for (callback = state-ctdb-script_callbacks; callback != NULL; 
callback = callback-next) {
+   if (callback == state-callback) {
+   break;
+   }
+   }
+   
+   state-callback = NULL;
 
if (callback) {
/* Make sure destructor doesn't free itself! */
@@ -669,8 +681,7 @@ static bool check_options(enum ctdb_eventscript_call call, 
const char *options)
 
 static int remove_callback(struct

[SCM] CTDB repository - branch 1.2.39 updated - ctdb-1.9.1-495-g1260c1b

2012-02-26 Thread Ronnie Sahlberg
The branch, 1.2.39 has been updated
   via  1260c1b9f301dccd497be8b818be5463858f0603 (commit)
   via  72dc4c9dc955489c312fc0866bf163132895d911 (commit)
  from  2f73114265b6523d6ceed54e70fd042c3738c6a2 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=1.2.39


- Log -
commit 1260c1b9f301dccd497be8b818be5463858f0603
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Mon Feb 27 07:18:19 2012 +1100

Make KILLTCP structure a child of VNN so that it is freed at the same time
the referenced VNN structure is.

Also, remove the circular reference between the two objects KIPPCTP and VNN

commit 72dc4c9dc955489c312fc0866bf163132895d911
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Wed Feb 22 17:38:12 2012 +1100

Eventscripts: remove the horrible horrible circular reference between state 
and callback since these two structures do not even share the same parent 
talloc context.
Instead, tie them together via referencing a permanent linked list hung off 
the ctdb structure.

---

Summary of changes:
 include/ctdb_private.h |3 +++
 server/ctdb_takeover.c |   22 +++---
 server/eventscript.c   |   26 +++---
 3 files changed, 41 insertions(+), 10 deletions(-)


Changeset truncated at 500 lines:

diff --git a/include/ctdb_private.h b/include/ctdb_private.h
index 675ea49..e3a9a15 100644
--- a/include/ctdb_private.h
+++ b/include/ctdb_private.h
@@ -495,6 +495,9 @@ struct ctdb_context {
 
/* Used to defer db attach requests while in recovery mode */
struct ctdb_deferred_attach_context *deferred_attach;
+
+   /* list of event script callback functions that are active */
+   struct event_script_callback *script_callbacks;
 };
 
 struct ctdb_db_context {
diff --git a/server/ctdb_takeover.c b/server/ctdb_takeover.c
index b2e1a8d..9cbafaf 100644
--- a/server/ctdb_takeover.c
+++ b/server/ctdb_takeover.c
@@ -2880,9 +2880,25 @@ static void ctdb_tickle_sentenced_connections(struct 
event_context *ev, struct t
  */
 static int ctdb_killtcp_destructor(struct ctdb_kill_tcp *killtcp)
 {
-   if (killtcp-vnn) {
-   killtcp-vnn-killtcp = NULL;
+   struct ctdb_vnn *tmpvnn;
+
+   /* verify that this vnn is still active */
+   for (tmpvnn = killtcp-ctdb-vnn; tmpvnn; tmpvnn = tmpvnn-next) {
+   if (tmpvnn == killtcp-vnn) {
+   break;
+   }
+   }
+
+   if (tmpvnn == NULL) {
+   return 0;
}
+
+   if (killtcp-vnn-killtcp != killtcp) {
+   return 0;
+   }
+
+   killtcp-vnn-killtcp = NULL;
+
return 0;
 }
 
@@ -2937,7 +2953,7 @@ static int ctdb_killtcp_add_connection(struct 
ctdb_context *ctdb,
   a new structure
 */
if (killtcp == NULL) {
-   killtcp = talloc_zero(ctdb, struct ctdb_kill_tcp);
+   killtcp = talloc_zero(vnn, struct ctdb_kill_tcp);
CTDB_NO_MEMORY(ctdb, killtcp);
 
killtcp-vnn = vnn;
diff --git a/server/eventscript.c b/server/eventscript.c
index 17cc3d4..c42aaa4 100644
--- a/server/eventscript.c
+++ b/server/eventscript.c
@@ -26,6 +26,7 @@
 #include ../include/ctdb_private.h
 #include lib/tevent/tevent.h
 #include ../common/rb_tree.h
+#include lib/util/dlinklist.h
 
 static void ctdb_event_script_timeout(struct event_context *ev, struct 
timed_event *te, struct timeval t, void *p);
 
@@ -41,7 +42,8 @@ static void sigterm(int sig)
 
 /* This is attached to the event script state. */
 struct event_script_callback {
-   struct ctdb_event_script_state *state;
+   struct event_script_callback *next, *prev;
+   struct ctdb_context *ctdb;
 
/* Warning: this can free us! */
void (*fn)(struct ctdb_context *, int, void *);
@@ -611,8 +613,18 @@ static int event_script_destructor(struct 
ctdb_event_script_state *state)
}
 
/* This is allowed to free us; talloc will prevent double free anyway,
-* but beware if you call this outside the destructor! */
-   callback = state-callback;
+* but beware if you call this outside the destructor!
+* the callback hangs off a different context so we walk the list
+* of active callbacks until we find the one state points to.
+* if we cant find it it means the callback has been removed.
+*/
+   for (callback = state-ctdb-script_callbacks; callback != NULL; 
callback = callback-next) {
+   if (callback == state-callback) {
+   break;
+   }
+   }
+   
+   state-callback = NULL;
 
if (callback) {
/* Make sure destructor doesn't free itself! */
@@ -669,8 +681,7 @@ static bool check_options(enum ctdb_eventscript_call call, 
const char *options)
 
 static int remove_callback

[SCM] CTDB repository - branch master updated - ctdb-1.12-213-gd8b400d

2012-02-21 Thread Ronnie Sahlberg
The branch, master has been updated
   via  d8b400d76665f37ffd9de302eedcff9f23807225 (commit)
   via  d3542d43141c1548bab356fbdef34dfedb4f167f (commit)
  from  9b85aa1aa14091dc1de470a587f7c054b9e40078 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=master


- Log -
commit d8b400d76665f37ffd9de302eedcff9f23807225
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Tue Feb 21 19:02:36 2012 +1100

when shutting down ctdb, allow it 30 seconds instead of 10 before will -9 
the daemon

commit d3542d43141c1548bab356fbdef34dfedb4f167f
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Tue Feb 21 19:00:33 2012 +1100

Log the Received SHUTDOWN ... first in the shutown process instead of 
last.

Sometimes ctdb is termnated ifrom the initscript if the shutdown takes too 
long an no message is logged.

---

Summary of changes:
 config/ctdb.init  |2 +-
 server/ctdb_control.c |2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)


Changeset truncated at 500 lines:

diff --git a/config/ctdb.init b/config/ctdb.init
index 9b30b86..0b86344 100755
--- a/config/ctdb.init
+++ b/config/ctdb.init
@@ -313,7 +313,7 @@ stop() {
 while pkill -0 -f $ctdbd ; do
sleep 1
count=$(($count + 1))
-   [ $count -gt 10 ]  {
+   [ $count -gt 30 ]  {
echo -n $killing ctdbd 
pkill -9 -f $ctdbd
pkill -9 -f $CTDB_BASE/events.d/
diff --git a/server/ctdb_control.c b/server/ctdb_control.c
index c734588..36b76cd 100644
--- a/server/ctdb_control.c
+++ b/server/ctdb_control.c
@@ -308,6 +308,7 @@ static int32_t ctdb_control_dispatch(struct ctdb_context 
*ctdb,
return 0;
 
case CTDB_CONTROL_SHUTDOWN:
+   DEBUG(DEBUG_NOTICE,(Received SHUTDOWN command. Stopping CTDB 
daemon.\n));
ctdb_stop_recoverd(ctdb);
ctdb_stop_keepalive(ctdb);
ctdb_stop_monitoring(ctdb);
@@ -316,7 +317,6 @@ static int32_t ctdb_control_dispatch(struct ctdb_context 
*ctdb,
ctdb-methods-shutdown(ctdb);
}
ctdb_event_script(ctdb, CTDB_EVENT_SHUTDOWN);
-   DEBUG(DEBUG_NOTICE,(Received SHUTDOWN command. Stopping CTDB 
daemon.\n));
exit(0);
 
case CTDB_CONTROL_TAKEOVER_IPv4:


-- 
CTDB repository


[SCM] CTDB repository - branch master updated - ctdb-1.12-218-gd306c3c

2012-02-21 Thread Ronnie Sahlberg
The branch, master has been updated
   via  d306c3c9a53e012c412c96ab9743de6cd96826e0 (commit)
  from  0df433496f9c53a2c3759a62ebe355c35caff920 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=master


- Log -
commit d306c3c9a53e012c412c96ab9743de6cd96826e0
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Wed Feb 22 16:07:17 2012 +1100

change TDB to be valgrind nice

---

Summary of changes:
 lib/tdb/common/hash.c |2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)


Changeset truncated at 500 lines:

diff --git a/lib/tdb/common/hash.c b/lib/tdb/common/hash.c
index c07297e..a29ba4a 100644
--- a/lib/tdb/common/hash.c
+++ b/lib/tdb/common/hash.c
@@ -22,6 +22,8 @@
You should have received a copy of the GNU Lesser General Public
License along with this library; if not, see http://www.gnu.org/licenses/.
 */
+#define VALGRIND
+
 #include tdb_private.h
 
 /* This is based on the hash algorithm from gdbm */


-- 
CTDB repository


[SCM] CTDB repository - branch master updated - ctdb-1.12-207-g0fd3bf9

2012-02-20 Thread Ronnie Sahlberg
The branch, master has been updated
   via  0fd3bf919b1b8e5aaa98444c306c6770a6a3209f (commit)
   via  11dee7f3f881494cf5089d6c69fd40e74f07e670 (commit)
   via  303134cf10a08ce61954d5de9025d9bbcb5f75ef (commit)
  from  d3c54ae9cc83f74d15f40bbfff95404b270d2f80 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=master


- Log -
commit 0fd3bf919b1b8e5aaa98444c306c6770a6a3209f
Merge: d3c54ae9cc83f74d15f40bbfff95404b270d2f80 
11dee7f3f881494cf5089d6c69fd40e74f07e670
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Mon Feb 20 21:30:13 2012 +1100

Merge branch 'master' of 10.1.1.27:/shared/ctdb/ctdb-master

commit 11dee7f3f881494cf5089d6c69fd40e74f07e670
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Mon Feb 20 21:22:56 2012 +1100

ReadOnly: We can not use ctdb_ltdb_store from a client/child context since
it sometimes (for empty records) needs to be able to initiate traffic unde 
rhte daemon context.

This should furhter updated later to allow the use also from non-daemon 
context.

commit 303134cf10a08ce61954d5de9025d9bbcb5f75ef
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Mon Feb 20 21:13:46 2012 +1100

ReadOnly: Make sure we dont try to fast-vacuum records that are set for 
readonly delegation

---

Summary of changes:
 server/ctdb_ltdb_server.c |2 ++
 server/ctdb_persistent.c  |   33 +++--
 2 files changed, 25 insertions(+), 10 deletions(-)


Changeset truncated at 500 lines:

diff --git a/server/ctdb_ltdb_server.c b/server/ctdb_ltdb_server.c
index e5437b9..e699c2a 100644
--- a/server/ctdb_ltdb_server.c
+++ b/server/ctdb_ltdb_server.c
@@ -83,6 +83,8 @@ static int ctdb_ltdb_store_server(struct ctdb_db_context 
*ctdb_db,
 */
if (data.dsize != 0) {
keep = true;
+   } else if (header-flags  
(CTDB_REC_RO_HAVE_DELEGATIONS|CTDB_REC_RO_HAVE_READONLY)) {
+   keep = true;
} else if (ctdb_db-persistent) {
keep = true;
} else if (header-flags  CTDB_REC_FLAG_AUTOMATIC) {
diff --git a/server/ctdb_persistent.c b/server/ctdb_persistent.c
index dd8d479..0f4f4da 100644
--- a/server/ctdb_persistent.c
+++ b/server/ctdb_persistent.c
@@ -456,12 +456,12 @@ static int ctdb_persistent_store(struct 
ctdb_persistent_write_state *state)
}
 
for (i=0;im-count;i++) {
-   struct ctdb_ltdb_header oldheader;
-   struct ctdb_ltdb_header header;
+   struct ctdb_ltdb_header *oldheader;
+   struct ctdb_ltdb_header *header;
TDB_DATA key, data, olddata;
TALLOC_CTX *tmp_ctx = talloc_new(state);
 
-   rec = ctdb_marshall_loop_next(m, rec, NULL, header, key, 
data);
+   rec = ctdb_marshall_loop_next(m, rec, NULL, NULL, key, data);

if (rec == NULL) {
DEBUG(DEBUG_ERR,(Failed to get next record %d for 
db_id 0x%08x in ctdb_persistent_store\n,
@@ -469,29 +469,42 @@ static int ctdb_persistent_store(struct 
ctdb_persistent_write_state *state)
talloc_free(tmp_ctx);
goto failed;
}
+   header = (struct ctdb_ltdb_header *)data.dptr[0];
 
/* fetch the old header and ensure the rsn is less than the new 
rsn */
-   ret = ctdb_ltdb_fetch(state-ctdb_db, key, oldheader, tmp_ctx, 
olddata);
-   if (ret != 0) {
+   olddata = tdb_fetch(state-ctdb_db-ltdb-tdb, key);
+   if (olddata.dptr == NULL) {
DEBUG(DEBUG_ERR,(Failed to fetch old record for db_id 
0x%08x in ctdb_persistent_store\n,
 state-ctdb_db-db_id));
talloc_free(tmp_ctx);
goto failed;
}
+   if (olddata.dsize  sizeof(struct ctdb_ltdb_header)) {
+   DEBUG(DEBUG_ERR,(Not enough header for record for 
db_id 0x%08x in ctdb_persistent_store\n,
+   state-ctdb_db-db_id));
+   talloc_free(tmp_ctx);
+   free(olddata.dptr);
+   goto failed;
+   }
+   oldheader = (struct ctdb_ltdb_header *)olddata.dptr[0];
 
-   if (oldheader.rsn = header.rsn 
-   (olddata.dsize != data.dsize || 
-memcmp(olddata.dptr, data.dptr, data.dsize) != 0)) {
+   if (oldheader-rsn = header-rsn 
+  (olddata.dsize != data.dsize || 
+  memcmp(olddata.dptr[sizeof(struct ctdb_ltdb_header)],
+data.dptr[sizeof(struct ctdb_ltdb_header)],
+data.dsize - sizeof(struct ctdb_ltdb_header)) != 0

[SCM] CTDB repository - branch master updated - ctdb-1.12-211-g9b85aa1

2012-02-20 Thread Ronnie Sahlberg
The branch, master has been updated
   via  9b85aa1aa14091dc1de470a587f7c054b9e40078 (commit)
   via  9e372a08c40087e6b5335aa298e94d88273566a5 (commit)
   via  b3307d78fd15f446b423f8cdd1e403f89fbe8ac8 (commit)
   via  5990fe65a9cc37933ceff15d4cb2ab3b3a0addb6 (commit)
  from  0fd3bf919b1b8e5aaa98444c306c6770a6a3209f (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=master


- Log -
commit 9b85aa1aa14091dc1de470a587f7c054b9e40078
Merge: 0fd3bf919b1b8e5aaa98444c306c6770a6a3209f 
9e372a08c40087e6b5335aa298e94d88273566a5
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Tue Feb 21 07:12:50 2012 +1100

Merge branch 'master' of 10.1.1.27:/shared/ctdb/ctdb-master

commit 9e372a08c40087e6b5335aa298e94d88273566a5
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Tue Feb 21 07:03:44 2012 +1100

READONLY: only send a control to schedule fast-vacuuming from child context 
iff we have a connection open to the main daemon

there are some child processes where we do not create a connection to the 
main daemon (switch_from_server_to_client()) because it is expensive to set up 
and we normally might not need to talk to the daemon at all via a domainsocket.
but we might want to still call to ctdb_ltdb_store() from such chil 
processes.

commit b3307d78fd15f446b423f8cdd1e403f89fbe8ac8
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Tue Feb 21 06:54:09 2012 +1100

READONLY: dont schedule for fast vacuum deletion if any of the readonly 
record flags are set

commit 5990fe65a9cc37933ceff15d4cb2ab3b3a0addb6
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Tue Feb 21 06:49:18 2012 +1100

Revert ReadOnly: We can not use ctdb_ltdb_store from a client/child 
context since

This reverts commit 11dee7f3f881494cf5089d6c69fd40e74f07e670.

Try to solve this a different way so that ctdb_ltb_store() becomes safe to 
use also from child context

---

Summary of changes:
 client/ctdb_client.c  |4 +++-
 common/ctdb_util.c|1 +
 include/ctdb_private.h|3 +++
 server/ctdb_ltdb_server.c |5 +++--
 server/ctdb_persistent.c  |   33 ++---
 server/ctdb_vacuum.c  |   10 +-
 6 files changed, 29 insertions(+), 27 deletions(-)


Changeset truncated at 500 lines:

diff --git a/client/ctdb_client.c b/client/ctdb_client.c
index 0d0f2fe..1b41439 100644
--- a/client/ctdb_client.c
+++ b/client/ctdb_client.c
@@ -4123,7 +4123,9 @@ int switch_from_server_to_client(struct ctdb_context 
*ctdb, const char *fmt, ...
return -1;
}
 
-return 0;
+   ctdb-can_send_controls = true;
+
+   return 0;
 }
 
 /*
diff --git a/common/ctdb_util.c b/common/ctdb_util.c
index bb32b6a..dadaf18 100644
--- a/common/ctdb_util.c
+++ b/common/ctdb_util.c
@@ -346,6 +346,7 @@ pid_t ctdb_fork(struct ctdb_context *ctdb)
if (ctdb-do_setsched) {
ctdb_restore_scheduler(ctdb);
}
+   ctdb-can_send_controls = false;
}
return pid;
 }
diff --git a/include/ctdb_private.h b/include/ctdb_private.h
index f420e6c..272b94a 100644
--- a/include/ctdb_private.h
+++ b/include/ctdb_private.h
@@ -483,6 +483,9 @@ struct ctdb_context {
 
/* Used to defer db attach requests while in recovery mode */
struct ctdb_deferred_attach_context *deferred_attach;
+
+   /* if we are a child process, do we have a domain socket to send 
controls on */
+   bool can_send_controls;
 };
 
 struct ctdb_db_context {
diff --git a/server/ctdb_ltdb_server.c b/server/ctdb_ltdb_server.c
index e699c2a..8e183e8 100644
--- a/server/ctdb_ltdb_server.c
+++ b/server/ctdb_ltdb_server.c
@@ -83,7 +83,7 @@ static int ctdb_ltdb_store_server(struct ctdb_db_context 
*ctdb_db,
 */
if (data.dsize != 0) {
keep = true;
-   } else if (header-flags  
(CTDB_REC_RO_HAVE_DELEGATIONS|CTDB_REC_RO_HAVE_READONLY)) {
+   } else if (header-flags  
(CTDB_REC_RO_HAVE_DELEGATIONS|CTDB_REC_RO_HAVE_READONLY|CTDB_REC_RO_REVOKING_READONLY|CTDB_REC_RO_REVOKE_COMPLETE))
 {
keep = true;
} else if (ctdb_db-persistent) {
keep = true;
@@ -128,7 +128,8 @@ static int ctdb_ltdb_store_server(struct ctdb_db_context 
*ctdb_db,
if (keep) {
if ((data.dsize == 0) 
!ctdb_db-persistent 
-   (ctdb_db-ctdb-pnn == header-dmaster))
+   (ctdb_db-ctdb-pnn == header-dmaster) 
+   !(header-flags  
(CTDB_REC_RO_HAVE_DELEGATIONS|CTDB_REC_RO_HAVE_READONLY|CTDB_REC_RO_REVOKING_READONLY|CTDB_REC_RO_REVOKE_COMPLETE)))
{
schedule_for_deletion = true;
}
diff --git a/server/ctdb_persistent.c b/server/ctdb_persistent.c
index 0f4f4da..dd8d479

[SCM] CTDB repository - branch master updated - ctdb-1.12-204-gd3c54ae

2012-02-19 Thread Ronnie Sahlberg
The branch, master has been updated
   via  d3c54ae9cc83f74d15f40bbfff95404b270d2f80 (commit)
   via  4ea05fef886b0680d0ad0936c239a51fd75bf000 (commit)
  from  650bb17335876f3e91e76a47b748c9236d0f3045 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=master


- Log -
commit d3c54ae9cc83f74d15f40bbfff95404b270d2f80
Merge: 4ea05fef886b0680d0ad0936c239a51fd75bf000 
650bb17335876f3e91e76a47b748c9236d0f3045
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Mon Feb 20 09:54:52 2012 +1100

Merge branch 'master' of ssh://git.samba.org/data/git/ctdb

commit 4ea05fef886b0680d0ad0936c239a51fd75bf000
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Thu Feb 16 10:19:15 2012 +1100

TESTS: Add test for wipedb command

---

Summary of changes:
 tests/simple/71_ctdb_wipedb.sh |   68 
 1 files changed, 68 insertions(+), 0 deletions(-)
 create mode 100755 tests/simple/71_ctdb_wipedb.sh


Changeset truncated at 500 lines:

diff --git a/tests/simple/71_ctdb_wipedb.sh b/tests/simple/71_ctdb_wipedb.sh
new file mode 100755
index 000..3859661
--- /dev/null
+++ b/tests/simple/71_ctdb_wipedb.sh
@@ -0,0 +1,68 @@
+#!/bin/bash
+
+test_info()
+{
+cat EOF
+The command 'ctdb wipedb' is used to clear a database across the whole
+cluster.
+
+Prerequisites:
+
+* An active CTDB cluster with at least 2 active nodes.
+
+Steps:
+
+1. Verify that the status on all of the ctdb nodes is 'OK'.
+2. create a persistent test database
+3, add some records to node #0 and node #1
+4, perform wipedb on node #0 and verify the database is empty on both node 0 
and 1
+
+Expected results:
+
+* that 4 will result in empty database
+
+EOF
+}
+
+. ctdb_test_functions.bash
+
+ctdb_test_init $@
+
+set -e
+
+cluster_is_healthy
+
+try_command_on_node 0 $CTDB listnodes
+num_nodes=$(echo $out | wc -l)
+
+# create a temporary persistent database to test with
+echo create persistent test database persistent_test.tdb
+try_command_on_node -q 0 $CTDB_TEST_WRAPPER ctdb attach persistent_test.tdb 
persistent
+
+
+# 3,
+# add one record to node 0   key==ABC  data==ABC
+TDB=`try_command_on_node -v -q 0 $CTDB_TEST_WRAPPER ctdb getdbmap | grep 
persistent_test.tdb | sed -e s/.*path:// -e s/ .*//`
+echo store key(ABC) data(ABC) on node 0
+try_command_on_node -q 0 $CTDB_TEST_WRAPPER ctdb tstore $TDB 0x414243 
0x0700414243
+#
+# add one record to node 1   key==DEF  data==DEF
+TDB=`try_command_on_node -v -q 1 $CTDB_TEST_WRAPPER ctdb getdbmap | grep 
persistent_test.tdb | sed -e s/.*path:// -e s/ .*//`
+echo store key(DEF) data(DEF) on node 1
+try_command_on_node -q 1 $CTDB_TEST_WRAPPER ctdb tstore $TDB 0x444546 
0x0700444546
+
+
+# 4,
+echo wipe the persistent test database
+try_command_on_node -q 0 $CTDB_TEST_WRAPPER ctdb wipedb persistent_test.tdb
+echo force a recovery
+try_command_on_node -q 0 $CTDB_TEST_WRAPPER ctdb recover
+
+# check that the database is wiped
+num_records=$(try_command_on_node -v -pq 1 $CTDB_TEST_WRAPPER ctdb cattdb 
persistent_test.tdb | grep key | wc -l)
+[ $num_records != 0 ]  {
+echo BAD: we did not end up with an empty database
+exit 1
+}
+echo OK. database was wiped
+


-- 
CTDB repository


[SCM] CTDB repository - branch 1.2.40 updated - ctdb-1.9.1-536-g2213847

2012-02-19 Thread Ronnie Sahlberg
The branch, 1.2.40 has been updated
   via  22138474060cba8c3a9b8b863488655ff6926258 (commit)
  from  0a9484c20cb0d3cd58c0ffeabca81c7b9aeca12d (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=1.2.40


- Log -
commit 22138474060cba8c3a9b8b863488655ff6926258
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Mon Feb 20 12:12:56 2012 +1100

ReadOnly: Do not force all normal database to READONLY.
Leave ro disabled by default and activate later by explicit calls to ctdb 
setdbreadonly

---

Summary of changes:
 server/ctdb_ltdb_server.c |   28 
 1 files changed, 0 insertions(+), 28 deletions(-)


Changeset truncated at 500 lines:

diff --git a/server/ctdb_ltdb_server.c b/server/ctdb_ltdb_server.c
index 42c3c56..27d773b 100644
--- a/server/ctdb_ltdb_server.c
+++ b/server/ctdb_ltdb_server.c
@@ -964,34 +964,6 @@ again:
}
}
 
-   /* Assume all non-persistent databases support read only delegations */
-   if (!ctdb_db-persistent) {
-   ctdb_db-readonly = true;
-   }
-
-   if (ctdb_db-readonly) {
-   char *ropath;
-
-   ropath = talloc_asprintf(ctdb_db, %s.RO, ctdb_db-db_path);
-   if (ropath == NULL) {
-   DEBUG(DEBUG_CRIT,(Failed to asprintf the tracking 
database\n));
-   ctdb_db-readonly = false;
-   talloc_free(ctdb_db);
-   return -1;
-   }
-   ctdb_db-rottdb = tdb_open(ropath, 
- ctdb-tunable.database_hash_size, 
- TDB_NOLOCK|TDB_CLEAR_IF_FIRST|TDB_NOSYNC,
- O_CREAT|O_RDWR, 0);
-   if (ctdb_db-rottdb == NULL) {
-   DEBUG(DEBUG_CRIT,(Failed to open/create the tracking 
database '%s'\n, ropath));
-   ctdb_db-readonly = false;
-   talloc_free(ctdb_db);
-   return -1;
-   }
-   DEBUG(DEBUG_NOTICE,(OPENED tracking database : '%s'\n, 
ropath));
-   }
-
 
DLIST_ADD(ctdb-db_list, ctdb_db);
 


-- 
CTDB repository


[SCM] CTDB repository - branch 1.2.39 updated - ctdb-1.9.1-493-g2f73114

2012-02-15 Thread Ronnie Sahlberg
The branch, 1.2.39 has been updated
   via  2f73114265b6523d6ceed54e70fd042c3738c6a2 (commit)
   via  91028144aa281a8f2ee915048c7c35d6c8a1d8df (commit)
   via  d9579c2971659f1ebc218a339d28f15132dd5718 (commit)
  from  0a626cef2547bbd3ea5c80da5ee2fb65691e0d88 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=1.2.39


- Log -
commit 2f73114265b6523d6ceed54e70fd042c3738c6a2
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Tue Feb 14 12:21:09 2012 +1100

New version 1.2.39-2

This version contains one single change:
Fix ip reallocation for LCP2 where it sometimes resulted
in poor balancing of the addresses across the nodes.

S1032414

commit 91028144aa281a8f2ee915048c7c35d6c8a1d8df
Author: Martin Schwenke mar...@meltin.net
Date:   Tue Nov 1 20:52:57 2011 +1100

LCP IP allocation algorithm - try harder to find a candidate source node

There's a bug in LCP2.  Selecting the node with the highest imbalance
doesn't always work.  Some nodes can have a high imbalance metric
because they have a lot of IPs.  However, these nodes can be part of a
group that is perfectly balanced.  Nodes in another group with less
IPs might actually be imbalanced.

Instead of just trying the source node with the highest imbalance this
tries them in descending order of imbalance until it finds one where
an IP can be moved to another node.

Signed-off-by: Martin Schwenke mar...@meltin.net

commit d9579c2971659f1ebc218a339d28f15132dd5718
Author: Martin Schwenke mar...@meltin.net
Date:   Tue Nov 1 19:49:38 2011 +1100

LCP IP allocation algorithm - new function lcp2_failback_candidate()

There's a bug in LCP2.  Selecting the node with the highest imbalance
doesn't always work.  Some nodes can have a high imbalance metric
because they have a lot of IPs.  However, these nodes can be part of a
group that is perfectly balanced.  Nodes in another group with less
IPs might actually be imbalanced.

Factor out the code from lcp2_failback() that actually takes a node
and decides which address should be moved to which node.

This is the first step in fixing the above bug.

Signed-off-by: Martin Schwenke mar...@meltin.net

---

Summary of changes:
 packaging/RPM/ctdb.spec.in |4 +-
 server/ctdb_takeover.c |  150 ++--
 2 files changed, 106 insertions(+), 48 deletions(-)


Changeset truncated at 500 lines:

diff --git a/packaging/RPM/ctdb.spec.in b/packaging/RPM/ctdb.spec.in
index 08719d0..7dd61bf 100644
--- a/packaging/RPM/ctdb.spec.in
+++ b/packaging/RPM/ctdb.spec.in
@@ -4,7 +4,7 @@ Summary: Clustered TDB
 Vendor: Samba Team
 Packager: Samba Team sa...@samba.org
 Version: 1.2.39
-Release: 1GITHASH
+Release: 2GITHASH
 Epoch: 0
 License: GNU GPL version 3
 Group: System Environment/Daemons
@@ -144,6 +144,8 @@ development libraries for ctdb
 %{_libdir}/libctdb.a
 
 %changelog
+* Tue Feb 14 2012 : Version 1.2.39-2
+ - Fix for ip reallocation bug resulting in poor istribution of addresses
 * Thu Nov 17 2011 : Version 1.2.39
  - Handle canceled monitor events better, dont assume they are always 
status==OK
 * Mon Oct 17 2011 : Version 1.2.38
diff --git a/server/ctdb_takeover.c b/server/ctdb_takeover.c
index 5865a17..b2e1a8d 100644
--- a/server/ctdb_takeover.c
+++ b/server/ctdb_takeover.c
@@ -1607,57 +1607,26 @@ void lcp2_allocate_unassigned(struct ctdb_context *ctdb,
}
 }
 
-/* LCP2 algorithm for rebalancing the cluster.  This finds the source
- * node with the highest LCP2 imbalance, and then determines the best
- * IP/destination node combination to move from the source node.
+/* LCP2 algorithm for rebalancing the cluster.  Given a candidate node
+ * to move IPs from, determines the best IP/destination node
+ * combination to move from the source node.
  *
  * Not static, so we can easily link it into a unit test.
  */
-bool lcp2_failback(struct ctdb_context *ctdb,
-  struct ctdb_node_map *nodemap,
-  uint32_t mask,
-  struct ctdb_public_ip_list *all_ips,
-  uint32_t *lcp2_imbalances,
-  bool *newly_healthy)
-{
-   int srcnode, dstnode, mindstnode, i, num_newly_healthy;
-   uint32_t srcimbl, srcdsum, maximbl, dstimbl, dstdsum;
-   uint32_t minsrcimbl, mindstimbl, b;
+bool lcp2_failback_candidate(struct ctdb_context *ctdb,
+struct ctdb_node_map *nodemap,
+struct ctdb_public_ip_list *all_ips,
+int srcnode,
+uint32_t candimbl,
+uint32_t *lcp2_imbalances,
+bool *newly_healthy)
+{
+   int dstnode, mindstnode;
+   uint32_t srcimbl, srcdsum, dstimbl

[SCM] CTDB repository - branch master updated - ctdb-1.12-197-ga7e7fa2

2012-02-12 Thread Ronnie Sahlberg
The branch, master has been updated
   via  a7e7fa2169ee12f817336f6d03d5bb72df7ec020 (commit)
   via  f4d395165816f74839ed48860e3210e05bc16d3d (commit)
   via  f0bc1760c36c8589f58354c50b23880946cb57c6 (commit)
   via  3c2117e8e4156e0048fbd2c94546112f0c588522 (commit)
  from  496e8f8540f78606ef23ee8fa100eb2b2cf3f106 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=master


- Log -
commit a7e7fa2169ee12f817336f6d03d5bb72df7ec020
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Mon Feb 13 11:16:24 2012 +1100

READONLY: add test for creating and revoking delegations

commit f4d395165816f74839ed48860e3210e05bc16d3d
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Mon Feb 13 11:06:45 2012 +1100

READONLY: Update the fetch-readonly-once test tool to use CTDB_SOCKET if set

commit f0bc1760c36c8589f58354c50b23880946cb57c6
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Mon Feb 13 10:34:41 2012 +1100

READONLY: document the command to activate readonly record support

commit 3c2117e8e4156e0048fbd2c94546112f0c588522
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Mon Feb 13 10:27:59 2012 +1100

READONLY:  allow specifying the db name for setdbreadonly instead of just 
the hash

---

Summary of changes:
 doc/ctdb.1|  676 +++--
 doc/ctdb.1.html   |  163 
 doc/ctdb.1.xml|   11 +
 tests/simple/75_readonly_records_basic.sh |  163 +++
 tests/src/ctdb_fetch_readonly_once.c  |8 +-
 tools/ctdb.c  |   33 ++-
 6 files changed, 541 insertions(+), 513 deletions(-)
 create mode 100755 tests/simple/75_readonly_records_basic.sh


Changeset truncated at 500 lines:

diff --git a/doc/ctdb.1 b/doc/ctdb.1
index 55454f3..c75a658 100644
--- a/doc/ctdb.1
+++ b/doc/ctdb.1
@@ -1,218 +1,164 @@
-'\ t
-.\ Title: ctdb
-.\Author: [FIXME: author] [see http://docbook.sf.net/el/author]
-.\ Generator: DocBook XSL Stylesheets v1.75.2 http://docbook.sf.net/
-.\  Date: 12/23/2011
-.\Manual: CTDB - clustered TDB database
-.\Source: ctdb
-.\  Language: English
-.\
-.TH CTDB 1 12/23/2011 ctdb CTDB \- clustered TDB database
-.\ -
-.\ * Define some portability stuff
-.\ -
-.\ ~
-.\ http://bugs.debian.org/507673
-.\ http://lists.gnu.org/archive/html/groff/2009-02/msg00013.html
-.\ ~
-.ie \n(.g .ds Aq \(aq
-.el   .ds Aq '
-.\ -
-.\ * set default formatting
-.\ -
+.\ ** You probably do not want to edit this file directly **
+.\ It was generated using the DocBook XSL Stylesheets (version 1.69.1).
+.\ Instead of manually editing it, you probably should edit the DocBook XML
+.\ source for it and then use the DocBook XSL Stylesheets to regenerate it.
+.TH CTDB 1 02/13/2012 ctdb CTDB \- clustered TDB database
 .\ disable hyphenation
 .nh
 .\ disable justification (adjust text to left margin only)
 .ad l
-.\ -
-.\ * MAIN CONTENT STARTS HERE *
-.\ -
 .SH NAME
 ctdb \- clustered tdb database management utility
 .SH SYNOPSIS
-.HP \w'\fBctdb\ [\ OPTIONS\ ]\ COMMAND\ \.\.\.\fR\ 'u
-\fBctdb [ OPTIONS ] COMMAND \.\.\.\fR
-.HP \w'\fBctdb\fR\ 'u
+.HP 29
+\fBctdb [ OPTIONS ] COMMAND ...\fR
+.HP 5
 \fBctdb\fR [\-n\ node] [\-Y] [\-t\ timeout] [\-T\ timelimit] [\-?\ 
\-\-help] [\-\-usage] [\-d\ \-\-debug=INTEGER] [\-\-socket=filename] 
[\-\-print\-emptyrecords] [\-\-print\-datasize] [\-\-print\-lmaster] 
[\-\-print\-hash] [\-\-print\-recordflags]
 .SH DESCRIPTION
 .PP
-ctdb is a utility to view and manage a ctdb cluster\.
+ctdb is a utility to view and manage a ctdb cluster.
 .SH OPTIONS
-.PP
+.TP
 \-n pnn
-.RS 4
-This specifies the physical node number on which to execute the command\. 
Default is to run the command on the daemon running on the local host\.
+This specifies the physical node number on which to execute the command. 
Default is to run the command on the daemon running on the local host.
 .sp
-The physical node number is an integer that describes the node in the 
cluster\. The first node has physical node number 0\.
-.RE
-.PP
+The physical node number is an integer that describes the node in the cluster. 
The first node has physical node number 0.
+.TP
 \-Y
-.RS 4
-Produce output in machine readable form for easier parsing by scripts\. Not 
all commands support this option\.
-.RE

[SCM] CTDB repository - branch master updated - ctdb-1.12-193-g496e8f8

2012-02-10 Thread Ronnie Sahlberg
The branch, master has been updated
   via  496e8f8540f78606ef23ee8fa100eb2b2cf3f106 (commit)
  from  751ed46197661eb841042ab6a02855a51dd0b17c (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=master


- Log -
commit 496e8f8540f78606ef23ee8fa100eb2b2cf3f106
Author: Mathieu Parent math.par...@gmail.com
Date:   Sat Feb 4 14:32:06 2012 +0100

Load platform dependent objs instead of Linux one

This fix compilation on Gnu/Hurd and kFreeBSD

See also:

https://buildd.debian.org/status/fetch.php?pkg=ctdbarch=hurd-i386ver=1.12%2Bgit20120201-1stamp=132829

https://buildd.debian.org/status/fetch.php?pkg=ctdbarch=kfreebsd-amd64ver=1.12%2Bgit20120201-1stamp=1328214761

https://buildd.debian.org/status/fetch.php?pkg=ctdbarch=kfreebsd-i386ver=1.12%2Bgit20120201-1stamp=1328215875

---

Summary of changes:
 Makefile.in|2 +-
 tests/src/ctdb_test.c  |3 ---
 tests/src/ctdbd_test.c |3 ---
 3 files changed, 1 insertions(+), 7 deletions(-)


Changeset truncated at 500 lines:

diff --git a/Makefile.in b/Makefile.in
index 15da883..8000801 100755
--- a/Makefile.in
+++ b/Makefile.in
@@ -240,7 +240,7 @@ CTDBD_TEST_C = $(CTDB_SERVER_MOST_OBJ:.o=.c) 
tests/src/ctdbd_test.c
 CTDB_TEST_C = $(CTDB_CLIENT_OBJ:.o=.c) tools/ctdb.c tools/ctdb_vacuum.c  
tests/src/ctdb_test.c
 
 CTDB_TEST_OBJ =  @TALLOC_OBJ@ @TDB_OBJ@ \
-   @LIBREPLACEOBJ@ $(EXTRA_OBJ) @TEVENT_OBJ@ $(SOCKET_WRAPPER_OBJ)
+   @CTDB_SYSTEM_OBJ@ @LIBREPLACEOBJ@ $(EXTRA_OBJ) @TEVENT_OBJ@ 
$(SOCKET_WRAPPER_OBJ)
 
 tests/src/ctdb_takeover_tests.o: tests/src/ctdb_takeover_tests.c 
$(CTDBD_TEST_C)
 
diff --git a/tests/src/ctdb_test.c b/tests/src/ctdb_test.c
index 83322d0..0514f43 100644
--- a/tests/src/ctdb_test.c
+++ b/tests/src/ctdb_test.c
@@ -78,9 +78,6 @@ void tevent_loop_allow_nesting_foobar(struct tevent_context 
*ev);
 #include common/cmdline.c
 #include lib/util/debug.c
 #include common/rb_tree.c
-#ifdef _LINUX_ERRNO_H
-#include common/system_linux.c
-#endif
 #include common/system_common.c
 #include common/ctdb_logging.c
 
diff --git a/tests/src/ctdbd_test.c b/tests/src/ctdbd_test.c
index 1e72f48..19861ff 100644
--- a/tests/src/ctdbd_test.c
+++ b/tests/src/ctdbd_test.c
@@ -53,9 +53,6 @@ void ctdb_load_nodes_file(struct ctdb_context *ctdb) {}
 #include common/cmdline.c
 #include lib/util/debug.c
 #include common/rb_tree.c
-#ifdef _LINUX_ERRNO_H
-#include common/system_linux.c
-#endif
 #include common/system_common.c
 #include common/ctdb_logging.c
 


-- 
CTDB repository


[SCM] CTDB repository - branch master updated - ctdb-1.12-190-g3e5d2b6

2012-02-07 Thread Ronnie Sahlberg
The branch, master has been updated
   via  3e5d2b61fcb4d0062d2573d99724f630a4fc4622 (commit)
   via  c6433e43ad2481d84e8d9a1ee1c6e29b90423a86 (commit)
   via  25d797f61c3b5b7090fb662adde6ed9e434e8a61 (commit)
  from  70d22efdf104db1fc951263cedfbdb95b0b44342 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=master


- Log -
commit 3e5d2b61fcb4d0062d2573d99724f630a4fc4622
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Wed Feb 8 08:59:53 2012 +1100

TESTS: Add a test that verifies that RecoverPDBBySeqNum works as expected.

If this tunable is set to 1 AND the persistent database contains a record 
__db_sequence_number__ then we will recover the whole database as-is instead 
of doing the record by record recovery

commit c6433e43ad2481d84e8d9a1ee1c6e29b90423a86
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Tue Feb 7 12:14:57 2012 +1100

Ad test tool to store data and ctdb header into a local tdb file

commit 25d797f61c3b5b7090fb662adde6ed9e434e8a61
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Tue Feb 7 10:53:49 2012 +1100

TFETCH add verbose flag to print also the record ltdb heaer

---

Summary of changes:
 tests/simple/70_recoverpdbbyseqnum.sh |  232 +
 tools/ctdb.c  |   83 +++-
 2 files changed, 309 insertions(+), 6 deletions(-)
 create mode 100755 tests/simple/70_recoverpdbbyseqnum.sh


Changeset truncated at 500 lines:

diff --git a/tests/simple/70_recoverpdbbyseqnum.sh 
b/tests/simple/70_recoverpdbbyseqnum.sh
new file mode 100755
index 000..4b46cbb
--- /dev/null
+++ b/tests/simple/70_recoverpdbbyseqnum.sh
@@ -0,0 +1,232 @@
+#!/bin/bash
+
+test_info()
+{
+cat EOF
+The tunable RecoverPDBBySeqNum controls how we perform recovery
+on persistent databases.
+The default is that persistent databases are recovered exactly the same
+way as normal databases. That is that we recover record by record.
+
+If RecoverPDBBySeqNum is set to 1 AND if a record with the key
+__db_sequence_number__ can be found in the database, then instead we will
+perform the recovery by picking the copy of the database from the node
+that has the highest sequence number and ignore the content on all other
+nodes.
+
+
+Prerequisites:
+
+* An active CTDB cluster with at least 2 active nodes.
+
+Steps:
+
+1. Verify that the status on all of the ctdb nodes is 'OK'.
+2. create a persistent test database
+3. test that RecoveryPDBBySeqNum==0 and no seqnum record blends the database
+   during recovery
+4. test that RecoveryPDBBySeqNum==0 and seqnum record blends the database
+   during recovery
+5. test that RecoveryPDBBySeqNum==1 and no seqnum record blends the database
+   during recovery
+6. test that RecoveryPDBBySeqNum==1 and seqnum record does not blend the 
database
+   during recovery
+
+Expected results:
+
+* that 3,4,5 will blend the databases and that 6 will recovery the highest 
seqnum
+  database
+
+EOF
+}
+
+. ctdb_test_functions.bash
+
+ctdb_test_init $@
+
+set -e
+
+cluster_is_healthy
+
+try_command_on_node 0 $CTDB listnodes
+num_nodes=$(echo $out | wc -l)
+
+# create a temporary persistent database to test with
+echo create persistent test database persistent_test.tdb
+try_command_on_node -q 0 $CTDB_TEST_WRAPPER ctdb attach persistent_test.tdb 
persistent
+
+
+# set RecoverPDBBySeqNum=0
+echo setting RecoverPDBBySeqNum to 0
+try_command_on_node -q all $CTDB_TEST_WRAPPER ctdb setvar RecoverPDBBySeqNum 0
+
+
+
+# 3,
+# If RecoverPDBBySeqNum==0  and no __db_sequence_number__
+# recover record by record
+#
+# wipe database
+echo
+echo test that RecoverPDBBySeqNum==0 and no __db_sequence_number__ blends the 
database during recovery
+echo wipe the test database
+try_command_on_node -q 0 $CTDB_TEST_WRAPPER ctdb wipedb persistent_test.tdb
+
+# add one record to node 0   key==ABC  data==ABC
+TDB=`try_command_on_node -v -q 0 $CTDB_TEST_WRAPPER ctdb getdbmap | grep 
persistent_test.tdb | sed -e s/.*path:// -e s/ .*//`
+echo store key(ABC) data(ABC) on node 0
+try_command_on_node -q 0 $CTDB_TEST_WRAPPER ctdb tstore $TDB 0x414243 
0x0700414243
+#
+# add one record to node 1   key==DEF  data==DEF
+TDB=`try_command_on_node -v -q 1 $CTDB_TEST_WRAPPER ctdb getdbmap | grep 
persistent_test.tdb | sed -e s/.*path:// -e s/ .*//`
+echo store key(DEF) data(DEF) on node 1
+try_command_on_node -q 1 $CTDB_TEST_WRAPPER ctdb tstore $TDB 0x444546 
0x0700444546
+
+# force a recovery
+echo force a recovery
+try_command_on_node -q 0 $CTDB_TEST_WRAPPER ctdb recover
+
+# check that we now have both records on node 0
+num_records=$(try_command_on_node -v -pq 0 $CTDB_TEST_WRAPPER ctdb cattdb 
persistent_test.tdb | grep key | egrep ABC|DEF | wc -l)
+[ $num_records != 2 ]  {
+echo BAD: we did

[SCM] CTDB repository - branch master updated - ctdb-1.12-192-g751ed46

2012-02-07 Thread Ronnie Sahlberg
The branch, master has been updated
   via  751ed46197661eb841042ab6a02855a51dd0b17c (commit)
   via  b098bcf8007be63889aaed640a951b0eeaa9d191 (commit)
  from  3e5d2b61fcb4d0062d2573d99724f630a4fc4622 (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=master


- Log -
commit 751ed46197661eb841042ab6a02855a51dd0b17c
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Wed Feb 8 15:29:27 2012 +1100

ReadOnly: add per-database statistics to view how much delegations/revokes 
we have

commit b098bcf8007be63889aaed640a951b0eeaa9d191
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Wed Feb 8 13:42:30 2012 +1100

STATISTICS: add total counts for number of delegations and number of revokes

Everytime we give a delegation to another node we count this as one 
delegation.
If the same record is delegated to several nodes we count one for each node.

Everytime a record has all its delegations revoked we count this as one 
revoke.

---

Summary of changes:
 include/ctdb.h|   57 +
 include/ctdb_private.h|   11 
 include/ctdb_protocol.h   |   11 
 libctdb/control.c |   51 
 libctdb/sync.c|   18 ++
 server/ctdb_call.c|4 +++
 server/ctdb_control.c |4 +++
 server/ctdb_daemon.c  |2 +
 server/ctdb_ltdb_server.c |   17 +
 tests/src/libctdb_test.c  |   17 +
 tools/ctdb.c  |   52 +
 11 files changed, 244 insertions(+), 0 deletions(-)


Changeset truncated at 500 lines:

diff --git a/include/ctdb.h b/include/ctdb.h
index 8dbdd47..93224cb 100644
--- a/include/ctdb.h
+++ b/include/ctdb.h
@@ -510,6 +510,39 @@ bool ctdb_getpnn_recv(struct ctdb_connection *ctdb,
 
 
 /**
+ * ctdb_getdbstat_send - read statistics for a db
+ * @ctdb: the ctdb_connection from ctdb_connect.
+ * @destnode: the destination node (see below)
+ * @db_id:the database to collect the statistics from
+ * @callback: the callback when ctdb replies to our message (typesafe)
+ * @cbdata: the argument to callback()
+ *
+ * There are several special values for destnode, detailed in
+ * ctdb_protocol.h, particularly CTDB_CURRENT_NODE which means the
+ * local ctdbd.
+ */
+struct ctdb_request *
+ctdb_getdbstat_send(struct ctdb_connection *ctdb,
+uint32_t destnode,
+uint32_t db_id,
+ctdb_callback_t callback,
+void *cbdata);
+/**
+ * ctdb_getdbstat_recv - read an ctdb_getdbstat reply from ctdbd
+ * @ctdb: the ctdb_connection from ctdb_connect.
+ * @req: the completed request.
+ * @stat: a pointer to the *stat to fill in
+ *
+ * This returns false if something went wrong, or otherwise fills in **stats
+ * stats must be freed later by calling ctdb_free_dbstat();
+ */
+bool ctdb_getdbstat_recv(struct ctdb_connection *ctdb,
+struct ctdb_request *req,
+struct ctdb_db_statistics **stat);
+
+void ctdb_free_dbstat(struct ctdb_db_statistics *stat);
+
+/**
  * ctdb_check_message_handlers_send - check a list of message_handlers
  * if they are registered
  * message_handlers are registered on the daemon using the
@@ -876,6 +909,26 @@ bool ctdb_getpnn(struct ctdb_connection *ctdb,
 uint32_t *pnn);
 
 /**
+ * ctdb_getdbstat - read the db stat of a node (synchronous)
+ * @ctdb: the ctdb_connection from ctdb_connect.
+ * @destnode: the destination node (see below)
+ * @db_id:the database to collect the statistics from
+ * @stat: a pointer to the *stat to fill in
+ *
+ * There are several special values for destnode, detailed in
+ * ctdb_protocol.h, particularly CTDB_CURRENT_NODE which means the
+ * local ctdbd.
+ *
+ * This returns false if something went wrong, or otherwise fills in **stat
+ * stat must be freed later by calling ctdb_free_dbstat();
+ */
+bool ctdb_getdbstat(struct ctdb_connection *ctdb,
+   uint32_t destnode,
+   uint32_t db_id,
+   struct ctdb_db_statistics **stat);
+
+
+/**
  * ctdb_check_message_handlers - check a list of message_handlers (synchronous)
  * @ctdb: the ctdb_connection from ctdb_connect.
  * @destnode: the destination node (see below)
@@ -1092,6 +1145,10 @@ void ctdb_free_vnnmap(struct ctdb_vnn_map *vnnmap);
ctdb_getpnn_send((ctdb), (destnode),\
 ctdb_sendcb((cb), (cbdata)), (cbdata))
 
+#define ctdb_getdbstat_send(ctdb, destnode, db_id, cb, cbdata) \
+   ctdb_getdbstat_send((ctdb), (destnode), (db_id),\
+   ctdb_sendcb((cb), (cbdata)), (cbdata))
+
 #define ctdb_check_message_handlers_send(ctdb, destnode, num

[SCM] CTDB repository - branch master updated - ctdb-1.12-187-g70d22ef

2012-02-06 Thread Ronnie Sahlberg
The branch, master has been updated
   via  70d22efdf104db1fc951263cedfbdb95b0b44342 (commit)
  from  4fa3f109b7ddc40e79c5fbb9688e0e830c0a655e (commit)

http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=master


- Log -
commit 70d22efdf104db1fc951263cedfbdb95b0b44342
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Tue Feb 7 10:21:12 2012 +1100

CTDB: update tfetch so we can specify a hexadecimal string as key

---

Summary of changes:
 tools/ctdb.c |   55 +--
 1 files changed, 45 insertions(+), 10 deletions(-)


Changeset truncated at 500 lines:

diff --git a/tools/ctdb.c b/tools/ctdb.c
index 445d56c..8ef869b 100644
--- a/tools/ctdb.c
+++ b/tools/ctdb.c
@@ -111,6 +111,33 @@ static const char *pretty_print_flags(uint32_t flags)
return flags_str;
 }
 
+static int h2i(char h)
+{
+   if (h = 'a'  h = 'f') return h - 'a' + 10;
+   if (h = 'A'  h = 'F') return h - 'f' + 10;
+   return h - '0';
+}
+
+static TDB_DATA hextodata(TALLOC_CTX *mem_ctx, const char *str)
+{
+   int i, len;
+   TDB_DATA key = {NULL, 0};
+
+   len = strlen(str);
+   if (len  0x01) {
+   DEBUG(DEBUG_ERR,(Key specified with odd number of hexadecimal 
digits\n));
+   return key;
+   }
+
+   key.dsize = len1;
+   key.dptr  = talloc_size(mem_ctx, key.dsize);
+
+   for (i=0; i  len/2; i++) {
+   key.dptr[i] = h2i(str[i*2])  4 | h2i(str[i*2+1]);
+   }
+   return key;
+}
+
 /* Parse a nodestring.  Parameter dd_ok controls what happens to nodes
  * that are disconnected or deleted.  If dd_ok is true those nodes are
  * included in the output list of nodes.  If dd_ok is false, those
@@ -3273,7 +3300,6 @@ static int control_readkey(struct ctdb_context *ctdb, int 
argc, const char **arg
 
db_name = argv[0];
 
-
if (db_exists(ctdb, db_name, persistent)) {
DEBUG(DEBUG_ERR,(Database '%s' does not exist\n, db_name));
return -1;
@@ -3288,7 +3314,7 @@ static int control_readkey(struct ctdb_context *ctdb, int 
argc, const char **arg
 
key.dptr  = discard_const(argv[1]);
key.dsize = strlen((char *)key.dptr);
- 
+
h = ctdb_fetch_lock(ctdb_db, tmp_ctx, key, data);
if (h == NULL) {
printf(Failed to fetch record '%s' on node %d\n, 
@@ -3322,7 +3348,6 @@ static int control_writekey(struct ctdb_context *ctdb, 
int argc, const char **ar
 
db_name = argv[0];
 
-
if (db_exists(ctdb, db_name, persistent)) {
DEBUG(DEBUG_ERR,(Database '%s' does not exist\n, db_name));
return -1;
@@ -3337,7 +3362,7 @@ static int control_writekey(struct ctdb_context *ctdb, 
int argc, const char **ar
 
key.dptr  = discard_const(argv[1]);
key.dsize = strlen((char *)key.dptr);
- 
+
h = ctdb_fetch_lock(ctdb_db, tmp_ctx, key, data);
if (h == NULL) {
printf(Failed to fetch record '%s' on node %d\n, 
@@ -3348,7 +3373,7 @@ static int control_writekey(struct ctdb_context *ctdb, 
int argc, const char **ar
 
data.dptr  = discard_const(argv[2]);
data.dsize = strlen((char *)data.dptr);
- 
+
if (ctdb_record_store(h, data) != 0) {
printf(Failed to store record\n);
}
@@ -3451,6 +3476,7 @@ static int control_tfetch(struct ctdb_context *ctdb, int 
argc, const char **argv
const char *tdb_file;
TDB_CONTEXT *tdb;
TDB_DATA key, data;
+   TALLOC_CTX *tmp_ctx = talloc_new(NULL);
int fd;
 
if (argc  2) {
@@ -3461,15 +3487,23 @@ static int control_tfetch(struct ctdb_context *ctdb, 
int argc, const char **argv
 
tdb = tdb_open(tdb_file, 0, 0, O_RDONLY, 0);
if (tdb == NULL) {
-   DEBUG(DEBUG_ERR,(Failed to open TDB file %s\n, tdb_file));
+   fprintf(stderr, Failed to open TDB file %s\n, tdb_file);
return -1;
}
 
-   key.dptr  = discard_const(argv[1]);
-   key.dsize = strlen(argv[1]);
+   if (!strncmp(argv[1], 0x, 2)) {
+   key = hextodata(tmp_ctx, argv[1] + 2);
+   if (key.dsize == 0) {
+   fprintf(stderr, Failed to convert \%s\ into a 
TDB_DATA\n, argv[1]);
+   return -1;
+   }
+   } else {
+   key.dptr  = discard_const(argv[1]);
+   key.dsize = strlen(argv[1]);
+   }
data = tdb_fetch(tdb, key);
if (data.dptr == NULL || data.dsize  sizeof(struct ctdb_ltdb_header)) {
-   DEBUG(DEBUG_ERR,(Failed to read record %s from tdb %s\n, 
argv[1], tdb_file));
+   fprintf(stderr, Failed to read record %s from tdb %s\n, 
argv[1], tdb_file);
tdb_close(tdb);
return -1;
}
@@ -3479,7 +3513,7 @@ static

[SCM] CTDB repository - branch martins created - 563cdf2bd9cca358d234c797d8e1d9c8838c6ca6

2012-02-05 Thread Ronnie Sahlberg
The branch, martins has been created
at  563cdf2bd9cca358d234c797d8e1d9c8838c6ca6 (commit)

- Log -
---


-- 
CTDB repository


[SCM] CTDB repository - branch 1.2.40 created - ctdb-1.9.1-535-g0a9484c

2012-02-05 Thread Ronnie Sahlberg
The branch, 1.2.40 has been created
at  0a9484c20cb0d3cd58c0ffeabca81c7b9aeca12d (commit)

- Log -
commit 0a9484c20cb0d3cd58c0ffeabca81c7b9aeca12d
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Mon Feb 6 09:22:37 2012 +1100

New version 1.2.40

commit a4888d22b56a149a323a7f1e50ddcb53a6fdd877
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Wed Feb 1 09:32:02 2012 +1100

ReadOnly: update the loop test tool to print number of fetches per second

commit f0274a3551ae984df8fb08d0c366e67e7ef6eb28
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Fri Oct 28 13:38:32 2011 +1100

libctdb: dont allow ctdb_writerecord() for readonly records

commit 91f5dcc5699a9ea9052c8d3c3c619c1d90c0cd58
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Fri Oct 28 12:41:27 2011 +1100

ReadOnly: If record does not exist, upgrade to write-lock

If we attempt a readonly lock request for a record that do not exist (yet)
in the local TDB, then upgrade the request to ask for a write lock and 
force a
request for migrate the record onto the local node.
This allows the only request record on second local request for known 
contended records
heuristics to try to avoid creating readonly delegations unless we have 
good reason to
assume it is a contended record.

commit 22e370afff7e1d236020071fa3d505c9fd3cfbd8
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Wed Feb 1 10:26:41 2012 +1100

ReadOnly: add readonly record lock requests to libctdb

Initial readonly record support in libctdb.
New records are not yet created by the library but extising records will be
This needs a bit more tests before we can drop the old style implementatio
code in client/ctdb_client.c

commit 2b07390620c16ba6515a37071bfc8c4e4b0d9ce9
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Fri Oct 28 11:44:19 2011 +1100

ReadOnly: fix bug writing incorrect amount of data in delegated record

Fix bug when ctdbd updates the local copy of a delegated record to write 
the correct
amount of data to the record.

commit 97e6d4c0eb0acb8543cb870943a8145f7f9eb499
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Mon Oct 24 13:19:30 2011 +1100

ReadOnly DOCS: update the docs for readonly delegations to remove the 
passage that records are written/updated by the client

commit c8ddb201ae1e399d45669d16be333aa9d744878c
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Mon Oct 24 13:14:26 2011 +1100

ReadOnly: Dont update the record header from the calling client. While it 
is convenient since it avoids having to create a child process from the main 
dameon for writing the updated record it makes the cleitn more complex.

Remove the code in the example client code that writes the record to the 
local tdb.
Add code to the local ctdbd processing of replies to check if this reply 
contain a ro delegation and if so, spawn a child process to lock the tdb and 
then write the data.

commit d63dc590e329ed553a4436a6555d91ad3d9fb363
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Tue Sep 13 18:47:18 2011 +1000

ReadOnly: revokechild_active is a list, not a context.

Dont reset the pointer to NULL after deleting the first entry, loop 
deleting one entry
at a time until they are all gone or we will leak some memory and possibly 
a process.

commit 70a7eebd580d1ff6d70528ec3890e9e9cc186ed5
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Tue Sep 13 18:41:34 2011 +1000

fix some compiler warnings for the test tools

commit 44f142cb93690acba6df5ce303d6df85bd5ae6f8
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Tue Sep 13 18:38:20 2011 +1000

ReadOnly: Rename the function ctdb_ltdb_fetch_readonly() to 
ctdb_ltdb_fetch_with_header() since this is what it actually does.

commit 459e3d7e26ec3cbbbdb7be7333c7e5ce70ea7f00
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Thu Sep 1 11:40:51 2011 +1000

ReadOnly: update the documentation about readonly locks

commit 9cf194a24ce5f8544ee989f07a6179a372e40b1e
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Thu Sep 1 11:08:18 2011 +1000

ReadOnly: add a new control to activate readonly lock capability for a 
database.
let all databases default to not support this  until enabled through this 
control

commit 96c4a06717a5792bf94b6c0c959117614a0e8b19
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Thu Sep 1 10:28:15 2011 +1000

ReadOnly: add a readonly flag to the getdbmap control and show the readonly 
setting in ctdb getdbmap output

commit 33c305e37fae6f804a33df6de581e9c4e0838d1e
Author: Ronnie Sahlberg ronniesahlb...@gmail.com
Date:   Thu Sep 1 10:21:55 2011 +1000

ReadOnly: Change the ctdb_db structure to keep a uint8_t for flags instead 
of a boolean for
the persistent flag.
This is the same size as the original boolean

  1   2   3   4   5   6   7   8   9   10   >