On 09/10/2013 06:26 PM, Lars Ellenberg wrote:
> On Mon, Sep 02, 2013 at 06:45:54PM +1000, Tim Serong wrote:
>> Hi All,
>>
>> I'm working on a build of csync2 2.0 for openSUSE, and have run into an
>> entertaining problem. Rather than linking against the relevant database
>> client libraries, it uses dlopen() at runtime. But it's dlopen()ing the
>> unversioned library, i.e.:
>>
>> db_mysql.c: dl_handle = dlopen("libmysqlclient.so", RTLD_LAZY);
>> db_postgres.c: dl_handle = dlopen("libpq.so", RTLD_LAZY);
>> db_sqlite.c: dl_handle = dlopen("libsqlite3.so", RTLD_LAZY);
>> db_sqlite2.c: dl_handle = dlopen("libsqlite.so", RTLD_LAZY);
>>
>> This is a problem for two reasons, one being that it means the -devel
>> packages need to be installed at runtime or those .so symlinks don't
>> exist. The other problem is that IMO it's just a bad idea to dlopen()
>> an unversioned library (potential runtime ABI breakage).
>>
>> Now, in the sqlite2 case, there's actually some fallback code, where if
>> it can't dlopen("libsqlite.so") it tries to dlopen("libsqlite.so.0").
>>
>> Personally I'd be in favour of ditching this fallback code and just
>> changing all the above to:
>>
>> db_mysql.c: dl_handle = dlopen("libmysqlclient.so.18", RTLD_LAZY);
>> db_postgres.c: dl_handle = dlopen("libpq.so.5", RTLD_LAZY);
>> db_sqlite.c: dl_handle = dlopen("libsqlite3.so.0", RTLD_LAZY);
>> db_sqlite2.c: dl_handle = dlopen("libsqlite.so.0", RTLD_LAZY);
>
> Ok.
>
> Thank you for the other patches as well.
> Did you hit any other issues,
> need to carry other patches?
We've got several other bits and pieces, which I'll attach to this email
for review.
* fix-sonames.patch gives the above versioned library names (although
I'd still be happier if they were somehow picked up/injected at build
time, rather than hard-coded).
* csync2-fix-xinetd.patch removes the need to add 30865 to /etc/services
and disables csync2 over xinetd by default.
* csync2.socket and csync2@.serivce allow running under systemd instead
of xinetd, and need to be installed in /usr/lib/systemd/system - I've
got these as additional source files referenced in our .spec, but it'd
be more useful if I wrote a patch to actually add them to the upstream
source tree proper.
* csync2-rm-ssl-cert is a shell script to remove stale SSL certificates
if you happen to replace or reinstall a node with a different SSL
certificate than has been used before. It's only useful with the
sqlite3 backend, so now that there's several DB backends I'm not sure
if this is desirable to carry, or if that functionality should maybe
be made a command line option to the csync2 binary itself instead.
* csync2-README.quickstart tells you how to get up and running in a
hurry, but note that it assumes SSL certs are already generated; we
have this happening in %post in the .spec file.
We did previously have a patch against 1.34 which wold force it to try
to bind to the IP address of the local hostname, then fallback to not
binding if that failed, but I note this has been handled in 2.0 with the
addition of the -N option.
How much of the above are you interested in taking? I'm happy to rework
it as proper patches to the git source if desirable.
Regards,
Tim
--
Tim Serong
Senior Clustering Engineer
SUSE
tser...@suse.com
Index: csync2-2.0+git.1368794815.cf835a7/db_mysql.c
===================================================================
--- csync2-2.0+git.1368794815.cf835a7.orig/db_mysql.c
+++ csync2-2.0+git.1368794815.cf835a7/db_mysql.c
@@ -53,16 +53,16 @@ static void *dl_handle;
static void db_mysql_dlopen(void)
{
- csync_debug(2, "Opening shared library libmysqlclient.so\n");
- dl_handle = dlopen("libmysqlclient.so", RTLD_LAZY);
+ csync_debug(2, "Opening shared library libmysqlclient.so.18\n");
+ dl_handle = dlopen("libmysqlclient.so.18", RTLD_LAZY);
if (dl_handle == NULL) {
csync_fatal
- ("Could not open libmysqlclient.so: %s\n"
+ ("Could not open libmysqlclient.so.18: %s\n"
"Please install Mysql client library (libmysqlclient) or use other database (sqlite, postgres)\n",
dlerror());
}
- csync_debug(2, "Reading symbols from shared library libmysqlclient.so\n");
+ csync_debug(2, "Reading symbols from shared library libmysqlclient.so.18\n");
LOOKUP_SYMBOL(dl_handle, mysql_init);
LOOKUP_SYMBOL(dl_handle, mysql_real_connect);
Index: csync2-2.0+git.1368794815.cf835a7/db_postgres.c
===================================================================
--- csync2-2.0+git.1368794815.cf835a7.orig/db_postgres.c
+++ csync2-2.0+git.1368794815.cf835a7/db_postgres.c
@@ -58,16 +58,16 @@ static void *dl_handle;
static void db_postgres_dlopen(void)
{
- csync_debug(2, "Opening shared library libpq.so\n");
+ csync_debug(2, "Opening shared library libpq.so.5\n");
- dl_handle = dlopen("libpq.so", RTLD_LAZY);
+ dl_handle = dlopen("libpq.so.5", RTLD_LAZY);
if (dl_handle == NULL) {
csync_fatal
- ("Could not open libpq.so: %s\n"
+ ("Could not open libpq.so.5: %s\n"
"Please install postgres client library (libpg) or use other database (sqlite, mysql)\n",
dlerror());
}
- csync_debug(2, "Reading symbols from shared library libpq.so\n");
+ csync_debug(2, "Reading symbols from shared library libpq.so.5\n");
LOOKUP_SYMBOL(dl_handle, PQconnectdb);
LOOKUP_SYMBOL(dl_handle, PQstatus);
Index: csync2-2.0+git.1368794815.cf835a7/db_sqlite.c
===================================================================
--- csync2-2.0+git.1368794815.cf835a7.orig/db_sqlite.c
+++ csync2-2.0+git.1368794815.cf835a7/db_sqlite.c
@@ -56,16 +56,16 @@ static void *dl_handle;
static void db_sqlite3_dlopen(void)
{
- csync_debug(2, "Opening shared library libsqlite3.so\n");
+ csync_debug(2, "Opening shared library libsqlite3.so.0\n");
- dl_handle = dlopen("libsqlite3.so", RTLD_LAZY);
+ dl_handle = dlopen("libsqlite3.so.0", RTLD_LAZY);
if (dl_handle == NULL) {
csync_fatal
- ("Could not open libsqlite3.so: %s\n"
+ ("Could not open libsqlite3.so.0: %s\n"
"Please install sqlite3 client library (libsqlite3) or use other database (postgres, mysql)\n",
dlerror());
}
- csync_debug(2, "Reading symbols from shared library libsqlite3.so\n");
+ csync_debug(2, "Reading symbols from shared library libsqlite3.so.0\n");
LOOKUP_SYMBOL(dl_handle, sqlite3_open);
LOOKUP_SYMBOL(dl_handle, sqlite3_close);
Index: csync2-2.0+git.1368794815.cf835a7/db_sqlite2.c
===================================================================
--- csync2-2.0+git.1368794815.cf835a7.orig/db_sqlite2.c
+++ csync2-2.0+git.1368794815.cf835a7/db_sqlite2.c
@@ -54,20 +54,17 @@ static void *dl_handle;
static void db_sqlite_dlopen(void)
{
- csync_debug(2, "Opening shared library libsqlite.so\n");
+ csync_debug(2, "Opening shared library libsqlite.so.0\n");
- dl_handle = dlopen("libsqlite.so", RTLD_LAZY);
+ dl_handle = dlopen("libsqlite.so.0", RTLD_LAZY);
if (dl_handle == NULL) {
- csync_debug(1, "Libsqlite.so not found, trying libsqlite.so.0\n");
- dl_handle = dlopen("libsqlite.so.0", RTLD_LAZY);
- if (dl_handle == NULL) {
csync_fatal
- ("Could not open libsqlite.so: %s\n"
+ ("Could not open libsqlite.so.0: %s\n"
"Please install sqlite client library (libsqlite) or use other database (postgres, mysql)\n",
dlerror());
}
}
- csync_debug(2, "Opening shared library libsqlite.so\n");
+ csync_debug(2, "Opening shared library libsqlite.so.0\n");
LOOKUP_SYMBOL(dl_handle, sqlite_open);
LOOKUP_SYMBOL(dl_handle, sqlite_close);
Index: csync2-2.0+git.1368794815.cf835a7/csync2.xinetd
===================================================================
--- csync2-2.0+git.1368794815.cf835a7.orig/csync2.xinetd
+++ csync2-2.0+git.1368794815.cf835a7/csync2.xinetd
@@ -1,4 +1,4 @@
-# default: on
+# default: off
# description: csync2
service csync2
{
@@ -9,7 +9,9 @@ service csync2
group = root
server = /usr/sbin/csync2
server_args = -i -l
+ port = 30865
+ type = UNLISTED
#log_on_failure += USERID
- disable = no
+ disable = yes
# only_from = 192.168.199.3 192.168.199.4
}
[Socket]
ListenStream=30865
Accept=yes
[Install]
WantedBy=sockets.target
[Unit]
Description=csync2 connection handler
After=syslog.target
[Service]
ExecStart=-/usr/sbin/csync2 -i -v
StandardInput=socket
StandardOutput=socket
#!/bin/bash
if [ $# -eq 0 -o "$1" = "-h" ]; then
cat <<END
Remove a peer's SSL certificate from csync2's local database. Use this after
replacing a peer node (or regenerating its SSL certificate).
Usage: $0 [-h] <hostname>
Options:
-h Display this usage information
END
exit 0
fi
DBFILE=/var/lib/csync2/$(hostname).db3
if [ ! -f "$DBFILE" ]; then
echo "Local csync2 database ($DBFILE) not found."
exit 1
fi
# Strip double and single quotes from hostname so they can't interfere with the
SQL
PEERNAME=$(echo $1 | sed -e "s/['\"]//g")
certcount()
{
echo "SELECT COUNT(peername) FROM x509_cert WHERE peername='$1';" |
sqlite3 $DBFILE
}
if [ $(certcount "$PEERNAME") -eq 0 ]; then
echo "Certificate for '$PEERNAME' not in local database."
exit 0
fi
echo "DELETE FROM x509_cert WHERE peername='$PEERNAME';" | sqlite3 $DBFILE
if [ $(certcount "$PEERNAME") -ne 0 ]; then
echo "Error removing certificate for '$PEERNAME' from local database."
exit 1
fi
echo "Certificate for '$PEERNAME' removed from local database."
Getting started with csync2 :
There's no need to define the port for csync2 in /etc/services, although the
manual says so. Port 30865/tcp is defined in /etc/xinetd.d/csync2. All commands
detailed here need to be executed as root, so be extra careful.
The config file for csync2 is /etc/csync2/csync2.cfg. Here is an example :
mygroup {
host host1;
host host2;
key /etc/csync2/mygroup.key;
include /etc/csync2/csync2.cfg;
include /etc/testfile;
}
This will sync the csync2 configuration and /etc/testfile between host1 and
host2. Create the file on host1. Note that hostnames need to be the FQDN
returned by "hostname".
Generate the pre-shared key used for authentication :
csync2 -k /etc/csync2/mygroup.key
Copy the configuration file and the pre-shared key to host2:
scp /etc/csync2/csync2.cfg /etc/csync2/mygroup.key host2:/etc/csync2/
The SSL key and certificate are generated upon package installation, but you
can replace them with your own if you like. The files are :
/etc/csync2/csync2_ssl_key.pem
/etc/csync2/csync2_ssl_cert.pem
Note that the common name (CN) in each node's SSL certificate must be the
same, or the SSL connection will fail. If you ever replace a node, and its
SSL key changes, existing nodes will still have a cached copy of the old key,
and the connection will fail. To remove the old key from an existing node's
cache, run the following command on each existing node:
csync2-rm-ssl-cert <replaced-hostname>
The csync2 service is disabled by default. To start it on both your hosts
using systemd (preferred):
systemctl enable csync2.socket
systemctl start csync2.socket
If you would prefer to do it the old way with xinetd, run:
chkconfig csync2 on
chkconfig --level 345 xinetd on
service xinetd restart
If you are running iptables, you need to open tcp port 30865 on both hosts so
that the other host can connect.
Now you should be able to run and initial verbose sync on both hosts :
csync2 -xv
Once everything looks good, you can add a file with the following line as
/etc/cron.d/csync2 or add it to /etc/crontab :
*/5 * * * * root csync2 -x
Happy syncing!
_______________________________________________
Csync2 mailing list
Csync2@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/csync2
