Re: De-duping attachments

2010-09-15 Thread Nik Conwell
Great thread. Here as some real world numbers based on our spools here at BU. One of our masters has 4,800 users, 22,000 mailboxes, and is using about 374G of disk. Based on the md5 files for these users there are 6,046,363 messages. If I look at the first md5 value (md5 on the msg if I

Re: choosing a file system

2008-12-31 Thread Nik Conwell
On Dec 30, 2008, at 4:43 PM, Shawn Nock wrote: [...] a scripted rename of mailboxes to balance partition utilization when we add another partition. Just curious - how do stop people from accessing their mailboxes during the time they are being renamed and moved to another partition?

Re: Archiving emails with Cyrus

2008-11-24 Thread Nik Conwell
. That way I can selectively gzip messages based on some algorithm involving message size and lack of activity. I haven't yet had the chance to do the analysis on the spool to see what sort of space gains I could expect. Nik Conwell Systems Programming Boston University Cyrus Home Page: http

Re: breaking into the system through cyrus account ?

2008-06-03 Thread Nik Conwell
On Jun 3, 2008, at 3:10 AM, Rudi Bruchez wrote: Hello, I'm using Cyrus on a Debian box, with pop3s. I found some time ago that someone was able to place a spamming tool in the /var/spool/cyrus/ directory. I cleaned it and changed all my passwords. All seemed ok. Hopefully you are keeping

quota bug involving nested quota roots?

2008-05-01 Thread Nik Conwell
lower quota root. If I remove the user/nik/restore quota and do quota -f, the value matches the original again: sq user/nik/restore remove 1 quota -f|grep -E user/nik|Quota Quota % Used Used Root 10485760 15 1640288 user/nik -nik Nik Conwell Office of Information Technology

Re: quota bug involving nested quota roots?

2008-05-01 Thread Nik Conwell
On Thu, 1 May 2008, Alain Spineux wrote: The full / 1024 matches the original 1640288 used so the problem seems to be quota -f not correctly traversing when there is another lower quota root. traversing ? Hum. Is it possible that cyrus stop counting as soon as it find another quota root ?

Re: Miserable performance of cyrus-imapd 2.3.9 -- seems to be locking issues

2008-02-29 Thread Nik Conwell
On Feb 28, 2008, at 4:38 PM, Jeff Fookson wrote: is about 200GB. There are typically about 200 'imapd' processes at a given time and a hugely varying number of 'lmtpds' (from about 6 to many hundreds during times of greatest pathology). System load is correspondingly in the 2-15

Re: tls self-signed certificates

2007-10-18 Thread Nik Conwell
On Oct 17, 2007, at 9:55 PM, Craig White wrote: OK - what I discovered was that TLS works with this setup (telnet localhost 143) IMAP/SSL doesn't seem to work when you 'telnet localhost 993' but on a client that is forgiving for self-signed certificates, it does actually work. So much

Re: UC Davis Cyrus Incident September 2007

2007-10-17 Thread Nik Conwell
for sharing Vincent. We are currently planning converting ~40K UW accounts to Cyrus. We have been a little more successful distributing the UW load so we are not quite as desperate as you were. :) -nik Nik Conwell Boston University Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ

Re: GSSAPI Murder authentication and The context has expired on long proxyd sessions

2007-09-12 Thread Nik Conwell
On Sep 11, 2007, at 3:00 PM, Paul M Fleming wrote: I had the same problems. if you google for this you'll find a discussion regarding how SASL context expires should be handled. Heimdal allows expired contexts to be used after expiration. MIT does not. Thanks. I had seen your posting

Re: tuning cyrus?

2007-08-27 Thread Nik Conwell
This is typically an Outlook problem. The client runs various filters, and possibly has perf issues on local disk (look for disk light activity on the PC) as it's updating its caches. Turn on Cyrus logging for the particular user (in the server config log subdir do mkdir username and make

Re: [INFO-CYRUS] Re: tuning cyrus?

2007-08-27 Thread Nik Conwell
On Aug 27, 2007, at 11:14 AM, Mike Eggleston wrote: [...] Where is the server logging sub-directory? I do not see one in /var/lib/cyrus-imapd (those are all binaries) nor do I see one in / var/log. It should be a subdirectory called log in whatever directory configdirectory is set to in

Re: Cyrus with a NFS storage. random DBERROR

2007-06-08 Thread Nik Conwell
it left off. -nik Nik Conwell Boston University Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: backup imapd with TSM

2007-05-24 Thread Nik Conwell
On Thu, 24 May 2007, Hans Moser wrote: Hi! Does anyone actually backup and restore Cyrus IMAPd with Tivoli Storage Manager (TSM)? As far as my backup admin told me, the restore tool only shows file names, I cannot see file content. So if user comes and tells I lost my email x from user

Cyrus 'cluster' general upgrade strategy?

2007-05-15 Thread Nik Conwell
discovered you can't replicate 2.3.7 to 2.3.8 (CREATE has an extra argument at 2.3.8) so it wouldn't surprise me if it generally isn't considered a good idea to mix versions like this in a murder. -nik Nik Conwell Information Technology Boston University [EMAIL PROTECTED

Re: Replication speeds?

2007-05-10 Thread Nik Conwell
On May 9, 2007, at 2:01 PM, Wesley Craig wrote: Obviously looking at more iostat information would give a better idea, but I'd estimate that you are NOT I/O bound. Sorry I can't give you absolute numbers from UM, but I can share a patch that we wrote that we believe has increase sync

Re: Recomendations for a 15000 Cyrus Mailboxes

2007-05-10 Thread Nik Conwell
On Apr 11, 2007, at 8:37 PM, Bron Gondwana wrote: As for complexity? It's on the cusp. We've certainly had many more users on a single instance before, but we prefer to keep under 10k users per Cyrus instance these days for quicker recoverability. It really Hi - just a clarification

Re: load balancing at fastmail.fm

2007-05-10 Thread Nik Conwell
On Jan 12, 2007, at 10:43 PM, Rob Mueller wrote: Yep, this means we need quite a bit more software to manage the setup, but now that it's done, it's quite nice and works well. For maintenance, we can safely fail all masters off a server in a few minutes, about 10-30 seconds a store. Then

Re: Replication and failover

2007-05-10 Thread Nik Conwell
On Jan 18, 2007, at 5:35 PM, Rob Mueller wrote: Attached is our operation group's notes on the subject. It makes reference to the tool we use to manage the OS of the machines (radmind), but it should be pretty clear what they are talking about without any radmind knowledge. As an FYI, we

Replication speeds?

2007-05-09 Thread Nik Conwell
/s rsec/s wsec/srkB/swkB/ s avgrq-sz avgqu-sz await svctm %util dm-0 0.00 0.00 0.00 429.960.00 3439.65 0.00 1719.83 8.00 7.24 16.75 1.86 80.01 Production (40,000+ users) would probably be FC and gigE... Nik Conwell Boston University [EMAIL

Bug? xfermailbox to murder front-end is broken but using rename to xfer a mailbox works just fine.

2007-05-07 Thread Nik Conwell
I can (via cyradm) issue renames to a frontend and move people from backend to backend just fine, but if I issue xfermailbox to the frontend to move people it hangs and I end up with a frontend mboxlist that's messed up (points to frontend). Is this a bug or am I just stupid issuing

BUG? File descriptor use in cmd_append (for MULTIAPPEND) results in many open files

2006-12-11 Thread Nik Conwell
I'm using the UW mailutil to transfer mailboxes from UW to Cyrus (2.3.7). It uses APPEND, specifically multiappend (single APPEND with multiple messages being appended). Cyrus-imapd handles this multiappend by creating stage files for each appended message and leaving the file

Re: BUG? File descriptor use in cmd_append (for MULTIAPPEND) results in many open files

2006-12-11 Thread Nik Conwell
On Dec 11, 2006, at 12:29 PM, Andrew Morgan wrote: On Mon, 11 Dec 2006, Nik Conwell wrote: I'm using the UW mailutil to transfer mailboxes from UW to Cyrus (2.3.7). It uses APPEND, specifically multiappend (single APPEND with multiple messages being appended). Cyrus-imapd handles

Re: BUG? File descriptor use in cmd_append (for MULTIAPPEND) results in many open files

2006-12-11 Thread Nik Conwell
On Dec 11, 2006, at 1:02 PM, Rich Graves wrote: Nik Conwell wrote: multiple messages being appended). Cyrus-imapd handles this multiappend by creating stage files for each appended message and leaving the file descriptor open. The problem is that after 240 messages, we run out of file

Re: BUG? File descriptor use in cmd_append (for MULTIAPPEND) results in many open files

2006-12-11 Thread Nik Conwell
On Dec 11, 2006, at 1:27 PM, Andrew Morgan wrote: On Mon, 11 Dec 2006, Nik Conwell wrote: On Dec 11, 2006, at 12:29 PM, Andrew Morgan wrote: On Mon, 11 Dec 2006, Nik Conwell wrote: I'm using the UW mailutil to transfer mailboxes from UW to Cyrus (2.3.7). It uses APPEND, specifically

Re: how to backup a cyrus server?

2006-12-05 Thread Nik Conwell
On Dec 4, 2006, at 6:42 PM, Andrew Morgan wrote: On Mon, 4 Dec 2006, Rafael Mahecha wrote: [...] I used to use tivoli to backup the old server (which was ok since no data bases were involved)... but since cyrus has databases and such, I am concern about file-locking and database

Re: how to backup a cyrus server?

2006-12-05 Thread Nik Conwell
] On Behalf Of Nik Conwell Sent: 05 December 2006 12:05 To: Cyrus User's Mailing List Subject: Re: how to backup a cyrus server? ctl_mboxlist -d for a text mailboxes file, ctl_cyrusdb -c to checkpoint, a sync, and then a lvcreate --snapshot --size 10G --name lv_cyrus_snapshot /dev/vg_cyrus/lv_cyrus

Re: timeouts when connecting to imap server

2006-12-01 Thread Nik Conwell
On Dec 1, 2006, at 12:29 PM, Timo Veith wrote: [...] Trying 127.0.0.1... Connected to 127.0.0.1. Escape character is '^]'. And I can wait and wait ... This is the point where I start wondering what the hell cyrus is doing now that it takes so long to answer. I started the master daemon

Re: Captive mailbox in Cyrus IMAP?

2006-11-30 Thread Nik Conwell
On Nov 29, 2006, at 4:17 PM, Greg A. Woods wrote: The only thing I don't like is use of a file in the filesystem as a flag instead of using a proper configuration file whos contents can be version controlled and more easily backed up, restored, documented, centralized, shared, and

Re: Captive mailbox in Cyrus IMAP?

2006-11-29 Thread Nik Conwell
Thanks to all for the suggestions. They were very good so I ignored them. :) I've patched imapd.c (cmd_login and cmd_authenticate) so that the presence and contents of {config_dir}/captive/{username} indicate the actual user that should be logged in (providing it begins with disabled).