A NOTE has been added to this issue. ====================================================================== http://dbmail.org/mantis/view.php?id=1081 ====================================================================== Reported By: PeterS Assigned To: ====================================================================== Project: DBMail Issue ID: 1081 Category: Authentication layer Reproducibility: always Severity: major Priority: normal Status: new target: ====================================================================== Date Submitted: 10-Aug-16 23:27 CEST Last Modified: 04-Oct-16 20:47 CEST ====================================================================== Summary: DBmail ABEND'ing upon LDAP access error. Description: I've been trying to find a version of DBmail 3.X that does not ABEND while it is accessing LDAP. I have not been able to determine the sequence of events that causes the ABEND, however it happens to me many many times a day. I've written a script around the execution of DBmail to ensure that it restarts when it ABENDs. ======================================================================
---------------------------------------------------------------------- (0003740) PeterS (reporter) - 11-Aug-16 19:57 http://dbmail.org/mantis/view.php?id=1081#c3740 ---------------------------------------------------------------------- Persuant to the "Bug posting etiquette", yesterday I pulled a clean, newest, version of DBmail from GIT, compiled it, installed it, cleared the dbmail.err log, and started it. Since then I have received 14 ABENDs. [compiler@swlx143 dbmail-git]$ date ; git status -u no Thu Aug 11 13:02:11 CDT 2016 # On branch master nothing to commit (working directory clean) [compiler@swlx143 dbmail-git]$ ---------------------------------------------------------------------- (0003745) PeterS (reporter) - 07-Sep-16 23:37 http://dbmail.org/mantis/view.php?id=1081#c3745 ---------------------------------------------------------------------- Today I migrated my database over to new hardware and am receiving the same error. dbmail-imapd: sasl.c:75: ldap_sasl_bind: Assertion `( (ld)->ldc->ldc_options.ldo_valid == 0x2 )' failed. Red Hat RHEL7 dbmail-auth-ldap-3.2.3-1.el7.x86_64 dbmail-3.2.3-1.el7.x86_64 postgresql-9.2.15-1.el7_2.x86_64 postgresql-server-9.2.15-1.el7_2.x86_64 ---------------------------------------------------------------------- (0003746) alan (reporter) - 08-Sep-16 19:01 http://dbmail.org/mantis/view.php?id=1081#c3746 ---------------------------------------------------------------------- This error appears to be ldap connection related. Error reports elsewhere suggest either something in /etc/openldap/ldap.conf or library compatibility following updates to openssl. Are there any certificates missing? Are you able to compile dbmail locally, this may address library issues. ---------------------------------------------------------------------- (0003749) PeterS (reporter) - 15-Sep-16 17:31 http://dbmail.org/mantis/view.php?id=1081#c3749 ---------------------------------------------------------------------- Alan, I have already tried the latest build(s) of DBmail to no avail. I've disabled all certificate handling in OPENLDAP using both: TLS_REQCERT never and also: TLS_REQCERT allow with no change in results. We have multiple Microsoft AD LDAP servers here, with varying certificates that aren't really verifiable, so I have always disabled tracking or requiring valid certificates. We have other Linux systems using LDAP/OPENLDAP without any issues using this same config. ---------------------------------------------------------------------- (0003751) alan (reporter) - 28-Sep-16 20:24 http://dbmail.org/mantis/view.php?id=1081#c3751 ---------------------------------------------------------------------- Confirm I'm able to reproduce the issue, think I've tracked down the parts of the code causing it. Will test asap. ---------------------------------------------------------------------- (0003752) alan (reporter) - 30-Sep-16 16:58 http://dbmail.org/mantis/view.php?id=1081#c3752 ---------------------------------------------------------------------- This issues appears to be fixed using the attached patch, please test. ---------------------------------------------------------------------- (0003753) PeterS (reporter) - 30-Sep-16 23:37 http://dbmail.org/mantis/view.php?id=1081#c3753 ---------------------------------------------------------------------- I've patched the Red Hat RHEL 7 (EPEL 7) version of DBmail (dbmail-3.2.3-1.el7.x86_64, dbmail-auth-ldap-3.2.3-1.el7.x86_64, and dbmail-debuginfo-3.2.3-1.el7.x86_64) with dbmail_src_modules_authldap.c-20160930.diff and receive the attached error, dbmail.20160930-1642.err . The service still ABEND's. ---------------------------------------------------------------------- (0003754) PeterS (reporter) - 01-Oct-16 00:18 http://dbmail.org/mantis/view.php?id=1081#c3754 ---------------------------------------------------------------------- I've patched a GIT clone (git clone git://git.dbmail.eu/paul/dbmail) from today with dbmail_src_modules_authldap.c-20160930.diff and receive the attached error, dbmail.20160930-1721.err . The service still ABEND's. ---------------------------------------------------------------------- (0003755) alan (reporter) - 01-Oct-16 15:36 http://dbmail.org/mantis/view.php?id=1081#c3755 ---------------------------------------------------------------------- Ok, latest patch src_modules_authldap.c-20161001.diff increases the time to check for server gone away to 2 minutes. There is an edge case where after binding, the connection instantly goes away. Also the daemon exits if the ldap server doesn't reappear after two minutes, similar to not starting if not available. There should be warnings such as the following, then recovers when the server returns. Oct 01 14:06:38 lully.p-o.co.uk dbmail-imapd[96422]: [0x805c0a800] Warning:[auth] ldap_con_get(+142): LDAP gone away: Can't contact LDAP server. Trying to reconnect(1/5). Oct 01 14:06:39 lully.p-o.co.uk dbmail-imapd[96422]: [0x805c0a800] Warning:[auth] ldap_con_get(+142): LDAP gone away: Can't contact LDAP server. Trying to reconnect(2/5). Oct 01 14:06:49 lully.p-o.co.uk dbmail-imapd[96422]: [0x805c0a800] Warning:[auth] ldap_con_get(+142): LDAP gone away: Can't contact LDAP server. Trying to reconnect(119/5). Does this get closer to addressing your issue? ---------------------------------------------------------------------- (0003756) PeterS (reporter) - 03-Oct-16 16:54 http://dbmail.org/mantis/view.php?id=1081#c3756 ---------------------------------------------------------------------- I've patched the Red Hat RHEL 7 (EPEL 7) version of DBmail (dbmail-3.2.3-1.el7.x86_64, dbmail-auth-ldap-3.2.3-1.el7.x86_64, and dbmail-debuginfo-3.2.3-1.el7.x86_64) with src_modules_authldap.c-20161001.diff and receive the attached error, dbmail.20161003-0956.err . This time I've cut the last parts of the full dbmail.err file which consists of the last threads 0x7f86e84e8000 and 0x7f86e81d1400 . The service still ABEND's. I'll now try with today's GIT version and the patch. ---------------------------------------------------------------------- (0003757) PeterS (reporter) - 03-Oct-16 19:57 http://dbmail.org/mantis/view.php?id=1081#c3757 ---------------------------------------------------------------------- I've patched a GIT clone (git clone git://git.dbmail.eu/paul/dbmail) from today with src_modules_authldap.c-20161001.diff and receive the attached error, dbmail.20161003-1055.err (dbmail.20161003-1055.err.xz) . This time I've cut the last parts of the full dbmail.err file which consists of the last threads 0x1466450 and 0xde7400. The service still ABEND's. ---------------------------------------------------------------------- (0003758) alan (reporter) - 04-Oct-16 17:54 http://dbmail.org/mantis/view.php?id=1081#c3758 ---------------------------------------------------------------------- Patch src_modules_authldap.c-20161004.diff adds slightly more debugging information, plus binding and search retries. The error appears to be transient and localised in authldap_search, this patch should address the issue. fyi there is a copy at https://github.com/alan-hicks/dbmail ---------------------------------------------------------------------- (0003759) PeterS (reporter) - 04-Oct-16 20:47 http://dbmail.org/mantis/view.php?id=1081#c3759 ---------------------------------------------------------------------- I pulled down a clone of your https://github.com/alan-hicks/dbmail and had it running. This version does not ABEND but rather stops communicating (which caused my client to Segfault in the middle of doing IMAPS things). Please see the attached dbmail.20161004-1348.err (dbmail.20161004-1348.err.xz) for the full error listing. Issue History Date Modified Username Field Change ====================================================================== 10-Aug-16 23:27 PeterS New Issue 10-Aug-16 23:27 PeterS File Added: dbmail.err 11-Aug-16 19:57 PeterS Note Added: 0003740 07-Sep-16 23:37 PeterS Note Added: 0003745 08-Sep-16 19:01 alan Note Added: 0003746 15-Sep-16 17:31 PeterS Note Added: 0003749 28-Sep-16 20:24 alan Note Added: 0003751 30-Sep-16 16:58 alan Note Added: 0003752 30-Sep-16 17:01 alan File Added: dbmail_src_modules_authldap.c.diff 30-Sep-16 18:15 alan Issue Monitored: alan 30-Sep-16 18:17 alan File Added: dbmail_src_modules_authldap.c-20160930.diff 30-Sep-16 23:37 PeterS Note Added: 0003753 30-Sep-16 23:37 PeterS File Added: dbmail.20160930-1642.err 01-Oct-16 00:18 PeterS Note Added: 0003754 01-Oct-16 00:18 PeterS File Added: dbmail.20160930-1721.err 01-Oct-16 15:29 alan File Added: src_modules_authldap.c-20161001.diff 01-Oct-16 15:36 alan Note Added: 0003755 03-Oct-16 16:54 PeterS Note Added: 0003756 03-Oct-16 16:56 PeterS File Added: dbmail.20161003-0956.err.xz 03-Oct-16 17:52 PeterS Note Added: 0003757 03-Oct-16 17:52 PeterS File Added: dbmail.20161003-1055.err.xz 03-Oct-16 19:57 PeterS Note Edited: 0003757 04-Oct-16 17:54 alan Note Added: 0003758 04-Oct-16 17:54 alan File Added: src_modules_authldap.c-20161004.diff 04-Oct-16 20:47 PeterS Note Added: 0003759 ====================================================================== _______________________________________________ Dbmail-dev mailing list Dbmail-dev@dbmail.org http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail-dev