Re: [courier-users] imapd seems to stall sometimes
On 25/05/14 11:04, Sam Varshavchik wrote: Leaving off MAXPERC should not be a factor. It defaults to, internally, to MAXDAEMONS – effectively a no-op. Right, thanks. I presume that you've eliminated the low hanging fruit of actually reaching the maximum number of connections. Yes, barely a dozen imapds running on the server with a MAXDAEMONS of 400 and I'd blame Thunderbird too but it wasn't happening 24 hours ago with Dovecot. An lsof of the gamin/gam_server daemon looks okay. A previous time I experienced something like this I had to change back to fam and remove gamin... years ago. Woops, yes, this is using Ubuntu 14.04 x64 (both ends)... courier-mta 0.68.2-1ubuntu3 courier-imap 4.10.0-20120615-1ubuntu3 courier-authlib-mysql0.63.0-6ubuntu1 Once MAXDAEMONS is reached, though, couriertcpd stops accepting connections altogether... Those are the kinds of things to look at, here. That is not the case in this situation. Here's a imap log dump from Thunderbird in case something is obvious... at the point below I clicked on my Junk folder... 1759508352[7fa967a50370]: proposed url = INBOX.Junk folder for connection INBOX has To Wait = FALSE can run = FALSE 895477504[7fa938decd00]: 35ffa800:renta.net:S-INBOX:SendData: DONE [.. then TB hangs here for more than a minute ...] 895477504[7fa938decd00]: ReadNextLine [stream=37b40c10 nb=0 needmore=1] 895477504[7fa938decd00]: 35ffa800:renta.net:S-INBOX:CreateNewLineFromSocket: clearing IMAP_CONNECTION_IS_OPEN - rv = 804b000e 895477504[7fa938decd00]: 35ffa800:renta.net:S-INBOX:TellThreadToDie: close socket connection 895477504[7fa938decd00]: 35ffa800:renta.net:S-INBOX:CreateNewLineFromSocket: (null) 1759508352[7fa967a50370]: creating protocol instance to retry queued url:imap://ma...@renta.net@renta.net:143/select.Junk 1759508352[7fa967a50370]: retrying url:imap://ma...@renta.net@renta.net:143/select.Junk 1759508352[7fa967a50370]: 3303c800:renta.net:NA:SetupWithUrl: clearing IMAP_CONNECTION_IS_OPEN 895477504[7fa938decd00]: ImapThreadMainLoop leaving [this=35ffa800] 963634944[7fa9337ae450]: ImapThreadMainLoop entering [this=3303c800] 963634944[7fa9337ae450]: 3303c800:renta.net:NA:ProcessCurrentURL: entering 963634944[7fa9337ae450]: 3303c800:renta.net:NA:ProcessCurrentURL:imap://markc%40renta%2e...@renta.net:143/select%3E.Junk: = currentUrl 963634944[7fa9337ae450]: ReadNextLine [stream=37da8510 nb=256 needmore=0] 963634944[7fa9337ae450]: 3303c800:renta.net:NA:CreateNewLineFromSocket: * OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA IDLE ACL ACL2=UNION STARTTLS LOGINDISABLED] Courier-IMAP ready. Copyright 1998-2011 Double Precision, Inc. See COPYING for distribution information. 963634944[7fa9337ae450]: 3303c800:renta.net:NA:SendData: 1 STARTTLS 963634944[7fa9337ae450]: ReadNextLine [stream=37da8510 nb=37 needmore=0] 963634944[7fa9337ae450]: 3303c800:renta.net:NA:CreateNewLineFromSocket: 1 OK Begin SSL/TLS negotiation now. 963634944[7fa9337ae450]: 3303c800:renta.net:NA:SendData: 2 capability 963634944[7fa9337ae450]: ReadNextLine [stream=37da8510 nb=133 needmore=0] 963634944[7fa9337ae450]: 3303c800:renta.net:NA:CreateNewLineFromSocket: * CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA IDLE AUTH=PLAIN ACL ACL2=UNION 963634944[7fa9337ae450]: ReadNextLine [stream=37da8510 nb=27 needmore=0] 963634944[7fa9337ae450]: 3303c800:renta.net:NA:CreateNewLineFromSocket: 2 OK CAPABILITY completed 963634944[7fa9337ae450]: try to log in 963634944[7fa9337ae450]: IMAP auth: server caps 0x4C3325, pref 0x1006, failed 0x0, avail caps 0x1004 963634944[7fa9337ae450]: (GSSAPI = 0x100, CRAM = 0x2, NTLM = 0x10, MSN = 0x20, PLAIN = 0x1000, LOGIN = 0x2, old-style IMAP login = 0x4)auth external IMAP login = 0x2000 963634944[7fa9337ae450]: trying auth method 0x1000 963634944[7fa9337ae450]: got new password 963634944[7fa9337ae450]: IMAP: trying auth method 0x1000 963634944[7fa9337ae450]: PLAIN auth 963634944[7fa9337ae450]: 3303c800:renta.net:NA:SendData: 3 authenticate plain 963634944[7fa9337ae450]: ReadNextLine [stream=37da8510 nb=4 needmore=0] 963634944[7fa9337ae450]: 3303c800:renta.net:NA:CreateNewLineFromSocket: + 963634944[7fa9337ae450]: 3303c800:renta.net:NA:SendData: Logging suppressed for this command (it probably contained authentication information) 963634944[7fa9337ae450]: ReadNextLine [stream=37da8510 nb=16 needmore=0] 963634944[7fa9337ae450]: 3303c800:renta.net:NA:CreateNewLineFromSocket: 3 OK LOGIN Ok. 963634944[7fa9337ae450]: login succeeded 963634944[7fa9337ae450]: 3303c800:renta.net:A:SendData: 4 select INBOX.Junk -- Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE Instantly run your Selenium tests across 300+ browser/OS combos. Get unparalleled scalability
Re: [courier-users] imapd seems to stall sometimes
Mark Constable writes: That is not the case in this situation. Here's a imap log dump from Thunderbird in case something is obvious... at the point below I clicked on my Junk folder... 1759508352[7fa967a50370]: proposed url = INBOX.Junk folder for connection INBOX has To Wait = FALSE can run = FALSE 895477504[7fa938decd00]: 35ffa800:renta.net:S-INBOX:SendData: DONE [.. then TB hangs here for more than a minute ...] 895477504[7fa938decd00]: ReadNextLine [stream=37b40c10 nb=0 needmore=1] 895477504[7fa938decd00]: 35ffa800:renta.net:S-INBOX:CreateNewLineFromSocket: clearing IMAP_CONNECTION_IS_OPEN - rv = 804b000e It's logging mostly its internal actions. Not 100% clear what's happening on the actual IMAP connection; but looks like it's in the middle of IDLE, it sends DONE, and claims that it doesn't get a response for a minute, times out, closes the connection, and creates the new one. That explains the delay. Need to set IMAPDEBUGFILE on the server side, and collect the actual IMAP traffic. This is already logged in context, that's for sure; so this is definitely not connection-oriented. So, IMAPDEBUGFILE would already have all the previous action recorded, so for clarity, need to truncate it to a 0-length file (not remove it), before repeating this action. Even better yet would be to find the server process, and strace it. Or, turn off IMAPENHANCEDIDLE, to see if that makes a difference. pgpUkxIQKAzx1.pgp Description: PGP signature -- Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE Instantly run your Selenium tests across 300+ browser/OS combos. Get unparalleled scalability from the best Selenium testing platform available Simple to use. Nothing to install. Get started now for free. http://p.sf.net/sfu/SauceLabs___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] imapd seems to stall sometimes
On 26/05/14 00:00, Sam Varshavchik wrote: Need to set IMAPDEBUGFILE on the server side, and collect the actual IMAP traffic. Got it, thanks, doing that now. Even better yet would be to find the server process, and strace it. It'll be tricky to find the right imapd process to attach to but I will try that after watching the imaplog.dat until I get a clue what to look for and/or pick up on where it, or TB, hangs. Or, turn off IMAPENHANCEDIDLE, to see if that makes a difference. No sign of that option on this server with courier-imap 4.10.0. Is that perhaps an option with a more recent version? -- Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE Instantly run your Selenium tests across 300+ browser/OS combos. Get unparalleled scalability from the best Selenium testing platform available Simple to use. Nothing to install. Get started now for free. http://p.sf.net/sfu/SauceLabs ___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] imapd seems to stall sometimes
On 26/05/14 00:00, Sam Varshavchik wrote: Or, turn off IMAPENHANCEDIDLE, to see if that makes a difference. Ah right, I see, it's IMAP_ENHANCEDIDLE. It was 0 so I just flipped it to 1 with IMAP_USELOCKS=1 and see what happens. Then I'll set them both to 0 as I rarely if ever use shared folders... and I don't think any clients even know of the possibility of using shard folders. Just now when clicking on a Junk folder, imaplog.dat showed... [...] 11 OK FETCH completed. WRITE: * BYE Disconnected for inactivity. WRITE: * BYE Disconnected for inactivity. [... long wait ...] WRITE: * BYE Disconnected for inactivity. [... shorter wait ...] WRITE: 3 OK LOGIN Ok. [...] etc, then the folder view in TB was suddenly refreshed on a new login. So something about being disconnected for inactivity then taking so long for a new login. I thought if I used IDLE then my connection would persist and re-logins, or at least not so many, would be the order of the day. I'm using 143/TLS with a real certificate. -- Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE Instantly run your Selenium tests across 300+ browser/OS combos. Get unparalleled scalability from the best Selenium testing platform available Simple to use. Nothing to install. Get started now for free. http://p.sf.net/sfu/SauceLabs ___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] imapd seems to stall sometimes
Mark Constable writes: On 26/05/14 00:00, Sam Varshavchik wrote: Or, turn off IMAPENHANCEDIDLE, to see if that makes a difference. Ah right, I see, it's IMAP_ENHANCEDIDLE. It was 0 so I just flipped it to 1 with IMAP_USELOCKS=1 and see what happens. Then I'll set them both to 0 as Well, turn off means 0, so keeping it at 0 would've kept things at their simplest. I rarely if ever use shared folders... and I don't think any clients even know of the possibility of using shard folders. Just now when clicking on a Junk folder, imaplog.dat showed... [...] 11 OK FETCH completed. WRITE: * BYE Disconnected for inactivity. WRITE: * BYE Disconnected for inactivity. [... long wait ...] WRITE: * BYE Disconnected for inactivity. [... shorter wait ...] WRITE: 3 OK LOGIN Ok. [...] etc, then the folder view in TB was suddenly refreshed on a new login. So something about being disconnected for inactivity then taking so long for a new login. I thought if I used IDLE then my connection would persist and re-logins, or at least not so many, would be the order of the day. Not quite. The default IMAP timeout is 30 minutes. IDLE eliminates periodic polling from the client for new mail, the server will notify the client when there's new mail. IMAP_ENHANCEDIDLE does not turn idle on, or off, it only enabled using FAM or Gamin for this, instead of the server polling internally. You indicate that imaplog.dat did not log the receipt of DONE from the client. This points to a network connection problem. Many cheap, consumer-grade NATing routers use very short timeouts to clear inactive NAT connections. My wireless router had a tendency to drop inactive ssh sessions after around 10 minutes, until I fiddled with ssh server's keepalive settings. With an IMAP client that uses IDLE, a persistent connection with no traffic of any kind on it, would've certainly gotten dropped. When the client, unaware that an intermediate router dropped the connection, tries to write something to it, if it's lucky it would get an an immediate connection RSET, and know that the connection is gone; but having the written data getting dropped quietly floor is certainly a plausible scenario. That's what it looks to me. So, the likeliest scenario here is that Thunderbird issues IDLE; not a byte crosses the persistent connection, in either direction, for 15-20 minutes, an intermediate NATting router drops the connection; Thunderbird wakes up, writes DONE; the same router just drops this rudely on the floor, since it no longer has any record of this connection, without sending back an RSET, Thunderbird spins for a minute and times out; on the server side, after 30 minutes have elapsed, the server logs that the client got disconnected for inactivity (the server also gets no indication that the connection was broken). Removing IDLE from the IMAP_CAPABILITY setting in the imapd configuration file will keep the server from advertising support for the IDLE extension altogether; that would be a rather crude way to work around this breakage. pgp6r1MDjcq51.pgp Description: PGP signature -- Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE Instantly run your Selenium tests across 300+ browser/OS combos. Get unparalleled scalability from the best Selenium testing platform available Simple to use. Nothing to install. Get started now for free. http://p.sf.net/sfu/SauceLabs___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] imapd seems to stall sometimes
Mark Constable writes: I was using ISPConfig3 + postfix for the past year and finally got around to unhitching myself from ISPConfig3 so I could run courier again on this particular server. However since the changeover I've noticed that Thunderbird seems to just hang and wait every half a dozen'th time when I go to check my mail. I don't recall this happening with Dovecot. Is it possible there is a MAXPERC setting missing like in esmptd? Leaving off MAXPERC should not be a factor. It defaults to, internally, to MAXDAEMONS – effectively a no-op. So, that can be pretty much eliminated as a suspect. I presume that you've eliminated the low hanging fruit of actually reaching the maximum number of connections. Additionally, it's been my experience that Thunderbird will report a new connection getting closed, and tell you to dial down an advanced setting for the maximum number of simultaneous connections. When MAXPERIP or MAXPERC is reached, the connection gets accepted and dropped, and Thunderbird will 'fess up, and play stupid. Once MAXDAEMONS is reached, though, couriertcpd stops accepting connections altogether. That would certainly result in a connection hang, until one existing connection gets disconnected. That would square with your symptoms, as well as anything else that might cause a hang after a connection is established; say you're using LDAP or MySQL as an authentication back-end, and the password validation query gets stalled, for some reason. Those are the kinds of things to look at, here. pgpJzNhn9k7Hu.pgp Description: PGP signature -- Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE Instantly run your Selenium tests across 300+ browser/OS combos. Get unparalleled scalability from the best Selenium testing platform available Simple to use. Nothing to install. Get started now for free. http://p.sf.net/sfu/SauceLabs___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
