Re: [Dovecot] 1.0.rc32 released
The LDAP_DEPRECATED define should have fixed these. Maybe your ldap.h doesn't have the deprecated functions at all for some reason? Or did I somehow mess up applying the deprecated-patch. :) rc32 builds cleanly for me (only a few minor warnings) and seems to be running well. Thanks! -Ben
Re: [Dovecot] Comment from OpenLDAP developer regarding "deferring operations"
On 4/12/07, Timo Sirainen <[EMAIL PROTECTED]> wrote: On Wed, 2007-04-11 at 19:50 -0500, Ben Beuchler wrote: > I'd note that if your client is performing search + bind on the same > connection in an asynchronous manner, then the server behavior as per > RFC4511 is undefined, since a server is not supposed to be willing to > handle operations intermixed with binds until a bind is concluded. That > is, any operation can be multiplexed on a single LDAP connection > __except__ binds, which must be serialized. That's how I thought it was working. But I think I found a bug in there. See if the attached patch fixes it? It does! Not a single "deferring operation" entry in nearly an hour! Now to wait and see if this change, Roy's patch, and linking against the new libldap_r can combine to eliminate my intermittent (and frustratingly non-reproducible) auth freezes. -Ben
Re: [Dovecot] Comment from OpenLDAP developer regarding "deferring operations"
I'll apply the patch later this morning and and let you know what I see. For what it's worth, last night I started dumping traffic between dovecot and openldap. The pattern I see most consistently was this: -> bind request (anonymous) -> search request <- bind response <- search result As I understand it, the search request being sent before the result of the bind request is received could trigger the deferrals. Thanks! -Ben On 4/12/07, Timo Sirainen <[EMAIL PROTECTED]> wrote: On Wed, 2007-04-11 at 19:50 -0500, Ben Beuchler wrote: > I'd note that if your client is performing search + bind on the same > connection in an asynchronous manner, then the server behavior as per > RFC4511 is undefined, since a server is not supposed to be willing to > handle operations intermixed with binds until a bind is concluded. That > is, any operation can be multiplexed on a single LDAP connection > __except__ binds, which must be serialized. That's how I thought it was working. But I think I found a bug in there. See if the attached patch fixes it?
[Dovecot] Comment from OpenLDAP developer regarding "deferring operations"
I received this response from a member of the OpenLDAP core team when asking about meaning of the deferred operations I was seeing. Is it possible that this is what I'm seeing, Timo? -Ben - I'd note that if your client is performing search + bind on the same connection in an asynchronous manner, then the server behavior as per RFC4511 is undefined, since a server is not supposed to be willing to handle operations intermixed with binds until a bind is concluded. That is, any operation can be multiplexed on a single LDAP connection __except__ binds, which must be serialized. A more appropriate client for that type of operations would need to use at least 2 connections, one for searches and one for binds, and the one for binds should take care of either performing synchronous binds, or in any case to wait for response to a bind before performing another. In case of failure (e.g. timeout) while waiting for bind response, the connection should be trashed (see ldap_unbind_ext(3)) and recreated. This is what is done, for example, by slapd-ldap(5). p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it --- Office: +39 02 23998309 Mobile: +39 333 4963172 Email: [EMAIL PROTECTED] ---
[Dovecot] Building Dovecot to link against alternative libldap
I'm running Dovecot on an Ubuntu (Edgy) box. I'd like to link dovecot against the latest stable version of the OpenLDAP libraries instead of the system defaults. I can't really replace the existing libraries as other packages rely on them. If this didn't use libtool, I would be able to influence the runtime linker by manipulating -rpath and -L. I can't seem to figure out how to get this to work with Dovecot's libtool-based build. The system libldap_r.so.2 lives in /usr/lib My libldap_r.so.2.3 lives in /usr/local/lib Anyone have any pointers? -Ben
Re: [Dovecot] rc31 -> "deferring operation: binding"
Some ideas from the peanut gallery here -- if you run 'ldd dovecot-auth', what openldap library is it linked into? The system libldap_r library, which is a 2.2.x release. Have you tried compiling dovecot against this new 2.3.32 openldap install? Nope; that's on my list of things to try today. Have you tried binding as version 2, just to see what happens? Worth trying. I'll give it a shot. -Ben
Re: [Dovecot] rc31 -> "deferring operation: binding"
Well, I've really no idea. I know it works for many people and I couldn't get it to break when stress testing with hundreds of connections per second. More info: In the 5 minutes between when Nagios flagged IMAP as down (and we tested it as down) until we restarted Dovecot, 103 authentications successfully completed. For the sake of reference, in the previous 5 minutes 311 IMAP logins succeeded. Also interesting: IMAP has died twice so far today (both times were at typical daily usage peaks). POP3 has not tested as down once. Admittedly, we do roughly 9 X as many IMAP logins as POP, but they both auth via dovecot-auth, right? -Ben
Re: [Dovecot] rc31 -> "deferring operation: binding"
The really frustrating thing, of course, is that I really have no proof whatsoever that the occasional auth failures (all authentication attempts hang until Dovecot is restarted) have anything to do with the Sorry; cut myself off. The really frustrating thing, of course, is that I really have no proof whatsoever that the occasional auth failures (all authentication attempts hang until Dovecot is restarted) have anything to do with the deferred binding issue. -Ben
Re: [Dovecot] rc31 -> "deferring operation: binding"
On 4/10/07, Timo Sirainen <[EMAIL PROTECTED]> wrote: On 10.4.2007, at 4.04, Ben Beuchler wrote: > As I understand it, the new auth_bind code is extremely careful about > flushing all pending operations before attempting to perform the > authentication bind. If that's the case, any idea why OpenLDAP is > deferring the binding operations so frequently? So far it hasn't > caused any login problems, but it's quite disconcerting given the > expected behavior described in the Changelog. Well, I've really no idea. I know it works for many people and I couldn't get it to break when stress testing with hundreds of connections per second. I installed a virgin install of OpenLDAP 2.3.32 directly on the mail server, copied over my LDAP database, pointed Dovecot at it, and started it up. I immediately started getting the same log entries: Apr 10 18:55:45 cliff slapd[10420]: connection_input: conn=3 deferring operation: binding I *must* be doing something wrong... Or, at least, I must be doing something a little different than the other test cases. Any thoughts? The really frustrating thing, of course, is that I really have no proof whatsoever that the occasional auth failures (all authentication attempts hang until Dovecot is restarted) have anything to do with the cliff:/usr/local/etc/openldap root# /opt/dovecot/sbin/dovecot -n # /opt/dovecot-1.0.rc31/etc/dovecot.conf log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap imaps pop3 pop3s ssl_cert_file: /etc/ssl/certs/mcad.edu.rapidssl.crt ssl_key_file: /etc/ssl/private/mcad.edu.rapidssl.key disable_plaintext_auth: no verbose_ssl: yes shutdown_clients: no login_dir: /opt/dovecot-1.0.rc31/var/run/dovecot/login login_executable(default): /opt/dovecot-1.0.rc31/libexec/dovecot/imap-login login_executable(imap): /opt/dovecot-1.0.rc31/libexec/dovecot/imap-login login_executable(pop3): /opt/dovecot-1.0.rc31/libexec/dovecot/pop3-login login_greeting: Line forms at the rear. login_greeting_capability(default): yes login_greeting_capability(imap): yes login_greeting_capability(pop3): no login_processes_count: 5 verbose_proctitle: yes first_valid_uid: 5000 last_valid_uid: 5000 first_valid_gid: 5000 last_valid_gid: 5000 mail_location: maildir:/mail/mcad.edu/%u/Maildir/ maildir_stat_dirs: yes maildir_copy_with_hardlinks: yes mail_executable(default): /opt/dovecot-1.0.rc31/libexec/dovecot/imap mail_executable(imap): /opt/dovecot-1.0.rc31/libexec/dovecot/imap mail_executable(pop3): /opt/dovecot-1.0.rc31/libexec/dovecot/pop3 mail_plugin_dir(default): /opt/dovecot-1.0.rc31/lib/dovecot/imap mail_plugin_dir(imap): /opt/dovecot-1.0.rc31/lib/dovecot/imap mail_plugin_dir(pop3): /opt/dovecot-1.0.rc31/lib/dovecot/pop3 pop3_uidl_format(default): pop3_uidl_format(imap): pop3_uidl_format(pop3): %08Xu%08Xv auth default: mechanisms: plain login user: dovecot-auth verbose: yes debug: yes passdb: driver: ldap args: /opt/dovecot/etc/dovecot-ldap.conf userdb: driver: static args: uid=5000 gid=5000 socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix master: cliff:~ root# egrep -v '(^$|^#)' /opt/dovecot/etc/dovecot-ldap.conf uris = ldap://localhost auth_bind = yes ldap_version = 3 base = dc=accounts,dc=ldap,dc=mcad,dc=edu pass_attrs = uid=user,userPassword=password pass_filter = (&(objectClass=mcadEmailAccount)(activePopImap=TRUE)(uid=%u)) default_pass_scheme = SSHA user_global_uid = 5000 user_global_gid = 5000
[Dovecot] "TLS handshake" errors, frozen IMAP
Another new issue has cropped up with my rc1 -> rc31 upgrade: This morning, right around the typical morning email surge, Dovecot stopped accepting new IMAP connections. I didn't get a chance to do much troubleshooting and a restart of Dovecot fixed it. The only thing I've been able to find in the logs is a bunch of entries like this one: Apr 10 09:11:36 cliff dovecot: imap-login: Disconnected: rip=150.253.80.90, lip=150.253.10.10, TLS handshake Apr 10 09:11:36 cliff dovecot: imap-login: Disconnected: rip=150.253.80.208, lip=150.253.10.10, TLS handshake Apr 10 09:11:37 cliff dovecot: imap-login: Disconnected: rip=150.253.91.156, lip=150.253.10.10, TLS handshake And a few like this: Apr 10 09:17:48 cliff dovecot: imap-login: Disconnected: Shutting down: rip=66.93.16.227, lip=150.253.10.10, TLS handshake Any idea what that might indicate? -Ben
Re: [Dovecot] rc31 -> "deferring operation: binding"
Is there anything else in slapd logs? I haven't been able to identify anything of consequence. For example I found this with google: http://www.openldap.org/lists/openldap-software/200507/ msg00063.html Aye, I saw that as well. Unfortunately, Howard points out that the author of that message was mistaken and, as near as I can tell, the only "solution" anyone could come up with was to upgrade. I'm already running the current stable OpenLDAP. I'll try to track down more info. -Ben
[Dovecot] rc31 -> "deferring operation: binding"
I hate to bring up anything that might delay 1.0, but the behavior I'm seeing is rather... weird. I'm running rc31, using "userdb static" and "auth_bind=yes". With rc1 (what I had been running) I would occasionally get some "deferring operation" complaints from LDAP, but very rarely. Every now and then it would start to throw "deferring operation: pending operations" and stop authenticating. Digging through the Changelog indicated you've made considerable updates to the auth_bind code, apparently to eliminate just such problems, so I upgraded. Now I'm running rc31 and my OpenLDAP logs are throwing an error like this every few seconds: Apr 9 19:56:42 swizzle slapd[1626]: connection_input: conn=6696168 deferring operation: binding This appears to happen with around every 3rd or 4th bind request. Sometimes it happens with the anonymous requests, sometimes with the auth bind request. As I understand it, the new auth_bind code is extremely careful about flushing all pending operations before attempting to perform the authentication bind. If that's the case, any idea why OpenLDAP is deferring the binding operations so frequently? So far it hasn't caused any login problems, but it's quite disconcerting given the expected behavior described in the Changelog. Any thoughts? -Ben cliff:/opt/dovecot/etc root# ../sbin/dovecot -n # /opt/dovecot-1.0.rc31/etc/dovecot.conf log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap imaps pop3 pop3s ssl_cert_file: /etc/ssl/certs/mcad.edu.rapidssl.crt ssl_key_file: /etc/ssl/private/mcad.edu.rapidssl.key disable_plaintext_auth: no shutdown_clients: no login_dir: /opt/dovecot-1.0.rc31/var/run/dovecot/login login_executable(default): /opt/dovecot-1.0.rc31/libexec/dovecot/imap-login login_executable(imap): /opt/dovecot-1.0.rc31/libexec/dovecot/imap-login login_executable(pop3): /opt/dovecot-1.0.rc31/libexec/dovecot/pop3-login login_greeting: Line forms at the rear. login_greeting_capability(default): yes login_greeting_capability(imap): yes login_greeting_capability(pop3): no login_processes_count: 5 verbose_proctitle: yes first_valid_uid: 5000 last_valid_uid: 5000 first_valid_gid: 5000 last_valid_gid: 5000 mail_location: maildir:/mail/mcad.edu/%u/Maildir/ maildir_stat_dirs: yes maildir_copy_with_hardlinks: yes mail_executable(default): /opt/dovecot-1.0.rc31/libexec/dovecot/imap mail_executable(imap): /opt/dovecot-1.0.rc31/libexec/dovecot/imap mail_executable(pop3): /opt/dovecot-1.0.rc31/libexec/dovecot/pop3 mail_plugin_dir(default): /opt/dovecot-1.0.rc31/lib/dovecot/imap mail_plugin_dir(imap): /opt/dovecot-1.0.rc31/lib/dovecot/imap mail_plugin_dir(pop3): /opt/dovecot-1.0.rc31/lib/dovecot/pop3 pop3_uidl_format(default): pop3_uidl_format(imap): pop3_uidl_format(pop3): %08Xu%08Xv auth default: mechanisms: plain login user: dovecot-auth passdb: driver: ldap args: /opt/dovecot/etc/dovecot-ldap.conf userdb: driver: static args: uid=5000 gid=5000 socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix master:
Re: [Dovecot] Time just moved backwards
Better time synchronization. Use ntpd to always keep your time in sync. It should never have to adjust by more than a few microseconds and Dovecot has no problem with that. -Ben On 4/8/07, Chaos Engine <[EMAIL PROTECTED]> wrote: Hi there, I got a daily cron (rdate to local time server) job wich adjusts time and which constantly gives me headache. Every day my dovecot suicides with: "dovecot: Time just moved backwards by 11 seconds. This might cause a lot of problems, so I'll just kill myself now." Of course my onboard clock is constantly off by more than 5 secs. I don't want to abandon time synchronization and I want to use dovecot. Maybe a -HUP signal would do? What do you propose? -- Chaos greets U
