Re: [Dovecot] lock files causing time outs with Thunderbird
On Thu, 2009-07-30 at 10:07 -0600, CJ Keist wrote: Okay, I think I got a test that can recreate the .lock file staying around so long. I have trash folder with about 3500 messages in it. I went in and deleted two messages from the Trash folder. How close to the end of the mailbox did you delete the messages from? I then clicked back to my inbox. There was a long pause where Thunderbird was saying Closing folder Then another long pause as it said Opening folder. After about two minutes thunderbird looks to have stopped processing and displayed my inbox. But the Trash.lock file stuck around for about another 5 minutes. Ran ptruss on the pid that still had the Trash folder open. There was no pid for the Trash.lock file during this time. What do you mean by this? Trash.lock didn't have a PID in it, but you found the PID anyway somehow? It looks to be doing seeks, stats, reads and writes over and over again. Attached is a partial listing of the ptruss command till the lock file went away. It looks like you deleted some messages over 4 MB from the end of file, and Dovecot just moves 4 MB data over the deleted one. It looks like it's being done in pretty inefficient way though.. I guess I should some day improve it, but that's probably going to be annoyingly difficult. Anyway, if you look at where most of the time is spent, it's in the pwrite64() calls. Many of them can take almost 0.1 seconds. Have you enabled synchronous writes or something? signature.asc Description: This is a digitally signed message part
Re: [Dovecot] lock files causing time outs with Thunderbird
The two message were at the end of the Trash folder. I used fuser to find the pids on the Trash and Trash.lock files. fuser reported no pid for the Trash.lock file. The the ptruss was run on the pid on the Trash file. By synchronous writes I'm assuming on the nfs mount options? No, they are async mounts. Timo Sirainen wrote: On Thu, 2009-07-30 at 10:07 -0600, CJ Keist wrote: Okay, I think I got a test that can recreate the .lock file staying around so long. I have trash folder with about 3500 messages in it. I went in and deleted two messages from the Trash folder. How close to the end of the mailbox did you delete the messages from? I then clicked back to my inbox. There was a long pause where Thunderbird was saying Closing folder Then another long pause as it said Opening folder. After about two minutes thunderbird looks to have stopped processing and displayed my inbox. But the Trash.lock file stuck around for about another 5 minutes. Ran ptruss on the pid that still had the Trash folder open. There was no pid for the Trash.lock file during this time. What do you mean by this? Trash.lock didn't have a PID in it, but you found the PID anyway somehow? It looks to be doing seeks, stats, reads and writes over and over again. Attached is a partial listing of the ptruss command till the lock file went away. It looks like you deleted some messages over 4 MB from the end of file, and Dovecot just moves 4 MB data over the deleted one. It looks like it's being done in pretty inefficient way though.. I guess I should some day improve it, but that's probably going to be annoyingly difficult. Anyway, if you look at where most of the time is spent, it's in the pwrite64() calls. Many of them can take almost 0.1 seconds. Have you enabled synchronous writes or something? -- C. J. Keist Email: cj.ke...@colostate.edu UNIX/Network ManagerPhone: 970-491-0630 Engineering Network ServicesFax: 970-491-5569 College of Engineering, CSU Ft. Collins, CO 80523-1301 All I want is a chance to prove 'Money can't buy happiness'
Re: [Dovecot] lock files causing time outs with Thunderbird
On Thu, 2009-07-30 at 11:56 -0600, CJ Keist wrote: The two message were at the end of the Trash folder. Is it possible that some older messages were also marked as \Deleted, and when you expunged the two at the end it also expunged something from the middle? (The truss really looks like something like that happened) I used fuser to find the pids on the Trash and Trash.lock files. fuser reported no pid for the Trash.lock file. The the ptruss was run on the pid on the Trash file. OK. The Trash.lock file isn't kept open, so that's why fuser didn't find it. By synchronous writes I'm assuming on the nfs mount options? No, they are async mounts. Actually now that I think of it, it's pretty unsafe if write is performed only locally without waiting for reply from NFS server that it got it. signature.asc Description: This is a digitally signed message part
Re: [Dovecot] lock files causing time outs with Thunderbird
On Fri, 2009-07-24 at 15:55 -0600, CJ Keist wrote: I have dovecot 1.1.16 running on Solaris 10. We are seeing problem with lock files taking forever to be removed after opening a email folder like Trash with several thousand messages in them. We use mbox format. You mean dotlock files (e.g. Trash.lock)? Does Dovecot log anything? Is there something I can do in the config file to speed things up? Are you using Dovecot deliver? If not, mbox_very_dirty_syncs=yes should speed things up with large mboxes. signature.asc Description: This is a digitally signed message part
Re: [Dovecot] lock files causing time outs with Thunderbird
Yes, I'm talking about Trash.lock files, not seeing any errors in the log files related with lock files. Yes, I am using dovecot deliver. So does that mean I should not set mbox_very_dirty_syncs to yes? Timo Sirainen wrote: On Fri, 2009-07-24 at 15:55 -0600, CJ Keist wrote: I have dovecot 1.1.16 running on Solaris 10. We are seeing problem with lock files taking forever to be removed after opening a email folder like Trash with several thousand messages in them. We use mbox format. You mean dotlock files (e.g. Trash.lock)? Does Dovecot log anything? Is there something I can do in the config file to speed things up? Are you using Dovecot deliver? If not, mbox_very_dirty_syncs=yes should speed things up with large mboxes. -- C. J. Keist Email: cj.ke...@colostate.edu UNIX/Network ManagerPhone: 970-491-0630 Engineering Network ServicesFax: 970-491-5569 College of Engineering, CSU Ft. Collins, CO 80523-1301 All I want is a chance to prove 'Money can't buy happiness'
Re: [Dovecot] lock files causing time outs with Thunderbird
On Fri, 2009-07-24 at 16:25 -0600, CJ Keist wrote: Yes, I'm talking about Trash.lock files, not seeing any errors in the log files related with lock files. Can you look at what Dovecot is doing while those .lock files exist? You can get the process pid from the .lock file and then do: truss -d -p pid And what process is it that's keeping those .lock files long? The same imap process that's trying to open the mailbox? Yes, I am using dovecot deliver. So does that mean I should not set mbox_very_dirty_syncs to yes? It means it shouldn't make a difference if it's set or not. But I guess you could try if enabling it happens to help for some reason.. signature.asc Description: This is a digitally signed message part
