On Sun, Aug 19, 2007 at 12:12:25AM -0700, Brian Lynch wrote: > In case anyone hits this, we finally tracked down the cause of the > errant behavior. Installing heartbeat prior to updating all software on > the box (yum update), caused a mismatch that resulted in an inability of > the linux-ha daemons to pick up the installed zlib or bz2 compression > libraries.
Do you know what did the update consist of? Normally, a bugfix update of a stable distribution shouldn't make big library changes. At any rate, ldd should be able to deal with this situation. Looking through your original post, I noticed that you didn't try ldd on dynamic libraries. Perhaps that would reveal something. When you say "installing" I guess that that also meant "compiling". That sounds like the most plausible place where a problem occured. Strange case indeed. > This resulted in the box not being able to join the cluster. > We put off the install of linux-ha until everything was up-to-date and > the problem didn't show up again. > > - Brian > > -----Original Message----- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Brian Lynch > Sent: Thursday, August 16, 2007 12:36 PM > To: General Linux-HA mailing list > Subject: RE: [Linux-HA] Error loading compression module > > Dejan, > Thanks for the response. I've run both an strace and ltrace on cib > and crmd with no appreciable errors. I verified both the rpm > installation and the actual size of the libraries between the working > set of hosts and the new broken hosts. I would agree with the > hypothesis of corruption, but have yet to turn up anything that would > point to the source of it. > > - Brian > > -----Original Message----- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Dejan > Muhamedagic > Sent: Thursday, August 16, 2007 10:43 AM > To: General Linux-HA mailing list > Subject: Re: [Linux-HA] Error loading compression module > > On Thu, Aug 16, 2007 at 06:53:57AM -0700, Brian Lynch wrote: > > Hi, > > > > > > > > We are having trouble moving our HA configuration into production. > Two > > boxes out of a cluster of 8 refuse to join before any CIB is loaded. > > The following errors occur on both hosts. > > > > > > > > Aug 15 20:23:42 afs00 cib: [21786]: ERROR: generic plugin load failed > > > > Aug 15 20:23:43 afs00 cib: [21786]: ERROR: get_compress_fns: loading > > compression module(zlib) failed > > > > Aug 15 20:23:43 afs00 cib: [21786]: ERROR: cl_decompress_field: > compress > > method(zlib) is not supported in this machine > > > > Aug 15 20:23:43 afs00 cib: [21786]: ERROR: compress2uncompress: > compress > > field failed > > > > Could you try strace? Or verify the relevant packages using rpm? > Must be some kind of corruption. > > Otherwise, I have no idea what could be the reason. I've never > seen anything like this. The plugin code is rather old and should > be bug free. > > > > > Note that we are running CentOS 5 with the following kernel revision: > > 2.6.18-8.1.8.el5.centos.plus. In our staging environment, we have > > identical hardware running a previous installation of CentOS 5 that > > works fine. > > > > > > > > Installed Packages > > > > Name : heartbeat > > > > Arch : x86_64 > > > > Version: 2.1.2 > > > > Release: 2.el5.centos > > > > Size : 6.5 M > > > > Repo : installed > > > > Summary: heartbeat - The Heartbeat Subsystem for High-Availability > Linux > > > > > > > > Also, zlib: > > > > > > > > Installed Packages > > > > Name : zlib > > > > Arch : x86_64 > > > > Version: 1.2.3 > > > > Release: 3 > > > > Size : 87 k > > > > Repo : installed > > > > Summary: The zlib compression and decompression library. > > > > > > > > And the physical libraries: > > > > > > > > lrwxrwxrwx 1 root root 13 Aug 15 00:20 /usr/lib64/libz.so.1 -> > > libz.so.1.2.3 > > > > -rwxr-xr-x 1 root root 85608 Jan 9 2007 /usr/lib64/libz.so.1.2.3 > > > > > > > > The cib finds them: > > > > > > > > afs01 tmp root # ldd /usr/lib64/heartbeat/cib > > > > libgnutls.so.13 => /usr/lib64/libgnutls.so.13 > > (0x00000030ca400000) > > > > libplumb.so.1 => /usr/lib64/libplumb.so.1 (0x0000003950400000) > > > > libpils.so.1 => /usr/lib64/libpils.so.1 (0x0000003950000000) > > > > libcrmcommon.so.1 => /usr/lib64/libcrmcommon.so.1 > > (0x0000003950c00000) > > > > libhbclient.so.1 => /usr/lib64/libhbclient.so.1 > > (0x0000003950800000) > > > > libcib.so.1 => /usr/lib64/libcib.so.1 (0x0000003951400000) > > > > libglib-2.0.so.0 => /lib64/libglib-2.0.so.0 > (0x00000032f1e00000) > > > > libcrypto.so.6 => /lib64/libcrypto.so.6 (0x00000032f3e00000) > > > > libccmclient.so.1 => /usr/lib64/libccmclient.so.1 > > (0x0000003951800000) > > > > libbz2.so.1 => /usr/lib64/libbz2.so.1 (0x00000030c7c00000) > > > > libz.so.1 => /usr/lib64/libz.so.1 (0x00000032f0e00000) > > > > libxml2.so.2 => /usr/lib64/libxml2.so.2 (0x00000032f8200000) > > > > libc.so.6 => /lib64/libc.so.6 (0x00000032efe00000) > > > > libuuid.so.1 => /lib64/libuuid.so.1 (0x00000030c7800000) > > > > libpam.so.0 => /lib64/libpam.so.0 (0x00000030c7400000) > > > > librt.so.1 => /lib64/librt.so.1 (0x00000032f1a00000) > > > > libdl.so.2 => /lib64/libdl.so.2 (0x00000032f0200000) > > > > libgcrypt.so.11 => /usr/lib64/libgcrypt.so.11 > > (0x00000030ca000000) > > > > libgpg-error.so.0 => /usr/lib64/libgpg-error.so.0 > > (0x00000032f1200000) > > > > libm.so.6 => /lib64/libm.so.6 (0x00000032f0600000) > > > > /lib64/ld-linux-x86-64.so.2 (0x00000032efa00000) > > > > libaudit.so.0 => /lib64/libaudit.so.0 (0x00000030c7000000) > > > > libpthread.so.0 => /lib64/libpthread.so.0 (0x00000032f0a00000) > > > > libnsl.so.1 => /lib64/libnsl.so.1 (0x00000030c9c00000) > > > > > > > > And the encryption libraries exist in plugins: > > > > > > > > afs01 tmp root # ls -l /usr/lib64/heartbeat/plugins/HBcompress > > > > total 48 > > > > -rw-r--r-- 1 root root 8992 Jul 31 15:25 bz2.a > > > > -rwxr-xr-x 1 root root 824 Jul 31 15:24 bz2.la > > > > -rwxr-xr-x 1 root root 6776 Jul 31 15:25 bz2.so > > > > -rw-r--r-- 1 root root 8938 Jul 31 15:25 zlib.a > > > > -rwxr-xr-x 1 root root 831 Jul 31 15:24 zlib.la > > > > -rwxr-xr-x 1 root root 6776 Jul 31 15:25 zlib.so > > > > > > > > I am attaching the ha.cf, the ha-debug log, and a listing of the > > yum-installed applications on the system. > > > > > > > > I found a few listings online from over year ago, but no resolutions: > > > > > > > > http://lists.linux-ha.org/pipermail/linux-ha/2006-February/018648.html > > > > > http://lists.community.tummy.com/pipermail/linux-ha/2006-October/022062. > > html > > > > > > > > Lastly, I attempted a complete rebuild of the systems and a downgrade > of > > heartbeat with no success. I'd like to avoid compiling heartbeat from > > scratch if possible. > > > > > > > > Thanks for any help! > > > > > > > > Best, > > > > Brian > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > Linux-HA mailing list > > [email protected] > > http://lists.linux-ha.org/mailman/listinfo/linux-ha > > See also: http://linux-ha.org/ReportingProblems > _______________________________________________ > Linux-HA mailing list > [email protected] > http://lists.linux-ha.org/mailman/listinfo/linux-ha > See also: http://linux-ha.org/ReportingProblems > _______________________________________________ > Linux-HA mailing list > [email protected] > http://lists.linux-ha.org/mailman/listinfo/linux-ha > See also: http://linux-ha.org/ReportingProblems > _______________________________________________ > Linux-HA mailing list > [email protected] > http://lists.linux-ha.org/mailman/listinfo/linux-ha > See also: http://linux-ha.org/ReportingProblems _______________________________________________ Linux-HA mailing list [email protected] http://lists.linux-ha.org/mailman/listinfo/linux-ha See also: http://linux-ha.org/ReportingProblems
