Re: Shared mailboxes doc

2011-01-09 Thread Shuvam Misra
On Fri, 2011-01-07 at 16:33 +, Andy Bennett wrote: Hi, Does anyone know the option that needs to be set (and how to set it) in order to do a bulletin board. i.e. have a separate SEEN state for each user? Do the imapd.conf sharedseenstate option (disabled) and setting

Re: GNU Mailutils

2010-12-20 Thread Shuvam Misra
Mailutils has been around since dirt, but I've never heard of it being used as a server. I do know that their IMAP support [client-side] is crap which is why everyone uses/used c-client [derived from UW]. The manual for the imap4d services [which runs under xinted, yuck!] is full of The

GNU Mailutils

2010-12-19 Thread Shuvam Misra
Hey guys, I know this is OT but any of you have any idea about this toolkit/system? I discovered it accidentally today --- didn't even know the GNU project had its own IMAP and POP3 servers in addition to client-side libraries. I wonder how good their IMAP daemon is, in terms of performance,

Re: Does anyone allow unlimited or extremely large quotas?

2010-11-16 Thread Shuvam Misra
I think the issue you will encounter first is clients will start to fall down when folders exceed a 'reasonable' number of messages. Common IMAP clients I've seen start to exhibit severe performance issues beyond a few hundred thousand messages. As far as I'm aware (the helpdesk guys

Re: Does anyone allow unlimited or extremely large quotas?

2010-11-16 Thread Shuvam Misra
Didn't Dave write up.imapproxy? It makes a huge difference for, e.g., IMP roundcube. Also, configuring client to not retrieve the LIST of mailboxes during every transaction is a big win. Thanks a lot -- will definitely incorporate it into our setup. How does one configure the client not to

Re: Does anyone allow unlimited or extremely large quotas?

2010-11-16 Thread Shuvam Misra
On 16.11.2010 17:11, Shuvam Misra wrote: I guess Webmail is OT on a Cyrus mailing list, but can't help asking: any suggestions for improving Webmail performance? (Admission: we haven't yet tried imapproxy -- it appears to be a good piece of C which will help things.) You should

Re: Does anyone allow unlimited or extremely large quotas?

2010-11-16 Thread Shuvam Misra
On 16 Nov 2010, at 10:32, Joseph Brennan wrote: I wish we'd somehow financed a native Cyrus webmail interface, that is not using IMAP but built into Cyrus. I don't think users know how good Cyrus is because they look at it through a weak intermediary. I don't think a Cyrus-specific web

Re: ext3 / XFS

2010-11-16 Thread Shuvam Misra
This is depends on what filesystem you are useing, I have mailboxes with hundreds of thousands of messages in them on XFS and have no problems, but on ext3 I start seeing slowdowns with a bit over ten thousand messages. Was dir_index enabled on that ext3 filesystem? Prior to

Re: replication

2010-11-12 Thread Shuvam Misra
Quoting Bron Gondwana br...@fastmail.fm: It's getting better, but it's still not 100% reliable to have master/master replication between two servers with interactions going to both sides. It SHOULD be safe now to have a single master/master setup with individual users on one side or

Re: IMAP not seeing old mail present on filesystem

2010-10-04 Thread Shuvam Misra
Dear Patrick, How would you do the IMAP append? Using a Perl::IMAP function? This isn't necessarily a concern for this list, but a few days ago I upgraded a site from cyrus 2.1.16 to cyrus 2.3.16 by using imapsync to transfer mail from the old server to the new one. This worked great

Re: competition

2010-09-26 Thread Shuvam Misra
For situations where we need just random access, not sequential, can we use GDBM? Is that library better than Berkeley DB? ^ G = GNU = GPL. Licencing issues I suspect. We're BSD licence, not GPL. Yes, you're quite right, I just checked. Till your comment, I had assumed that GDBM

Re: competition

2010-09-25 Thread Shuvam Misra
On 9/22/2010 10:20 PM, Shuvam Misra wrote: I was a strong advocate of bundling DB libraries, etc, with Cyrus. The points you've made here are very interesting. I didn't know many of these things. I'm re-thinking whether bundling is such a good idea now. Thanks. There's a lot to be said

Berkeley DB: private copy?

2010-09-22 Thread Shuvam Misra
given the issues with BDB. Is it worth embedding a copy of BDB into the Cyrus distribution rather than using the OS one? I know it's generally considered bad taste, but it sure makes keeping in sync easier! IMHO, yes, most certainly. Cyrus is a large and complex system, and its

Re: competition

2010-09-22 Thread Shuvam Misra
given the issues with BDB. Is it worth embedding a copy of BDB into the Cyrus distribution rather than using the OS one? I That way lies madness. BDB is one of those things where arcane blackmagic skills are needed to keep it working on all arches. It uses scary crap to be fast

Re: competition

2010-09-22 Thread Shuvam Misra
Kolab Systems is thinking of such SQL databases for integration purposes, where the performance penalty now lies within having to use the IMAP protocol to gain access to the underlying metadata (seen status, message indexes) in distributed groupware environments where Cyrus itself is not the

Is the statuscache.db patch part of Cyrus now?

2010-09-21 Thread Shuvam Misra
Is this patch: http://www.mail-archive.com/cyrus-de...@lists.andrew.cmu.edu/msg01194.html part of Cyrus now? We recently read a post from one friend who said he's using this patch as part of the optimisations for his large setup. regards, Shuvam Cyrus Home Page:

RE: A beginner question about Murder

2010-09-16 Thread Shuvam Misra
Dear Michel, we use a modified traditional murder, i.e. without murder daemon, to host more than 2 million mailboxes (dozen million entries in mboxlist with folders) Wow, that's some figure. Care to share some details with the list? What kind of hardware (both for servers and storage), what

RE: A beginner question about Murder

2010-09-16 Thread Shuvam Misra
So we use about ten common Intel based servers for BEs (sized to support loss of 2/3 servers) and store data on NAS. Users are active and filer is about tens of thousand nfsop/s at the busy hour. About 10% to 20% of the users connect at least one time a day, globally 4 million connections

Re: De-duping attachments

2010-09-15 Thread Shuvam Misra
Dear Bron, So you save, what, 50%. Does that sound about right? Do you have statistics on how much space you'd save with this theoretical patch? No, and this is the first thing I want to do. I'm getting some simple utilities developed which will run all week (niced suitably) and extract and

Re: De-duping attachments

2010-09-15 Thread Shuvam Misra
The sparse file idea is brilliant! Never occurred to me. :) We'd have to store the reference-pointer in the message file, so we would omit the actual attachment but eat up perhaps 50 bytes to keep the reference to the file. Shuvam 1. Completely rewrite the message file removing the

Re: Importing/moving an older cyrus message tree into a new system, without IMAP

2010-09-15 Thread Shuvam Misra
Annotations are defined in RFC 5257. They allow an admin to add metadata to a mailbox (or the server). The cyradm utility sets annotations with its internal info, mboxcfg, and setinfo commands. Okay, checked. Don't know where these things are used, other than expiry and sieve, but at least

Re: De-duping attachments

2010-09-15 Thread Shuvam Misra
Makes sense. There might be some size based logic here too - only bother applying this on messages over 20k, and where the attachment is at least 20k in size. Anything smaller than that is pretty pointless. Yes, absolutely. Left to myself, I'd not have bothered with any attachment less than

Re: Importing/moving an older cyrus message tree into a new system, without IMAP

2010-09-14 Thread Shuvam Misra
Dear Dan, If you're not concerned about your quota database, seen state, annotations, and subscription information, and assuming you've already regenerated your top level mailbox hierarchy, then you should be able to copy over the individual email files from each mailbox to the new server and

Re: Importing/moving an older cyrus message tree into a new system, without IMAP

2010-09-14 Thread Shuvam Misra
We did a migration some months back from an old Kolab v1 (cyrus v2.1) system to a new Kolab v2.2 (cyrus v2.2) system. This was done by writing a script to - dump the ldap database (you might not have this) and load it on the new system - rsync the mailboxes from their location on

Re: Importing/moving an older cyrus message tree into a new system, without IMAP

2010-09-14 Thread Shuvam Misra
Dear Dan, See the manpage for imapd.conf for possible formats, but for my 2.3.12 installation, with configdirectory specified at /var/lib/cyrus (and no customization to my *_db options), my database files are: Got it. Thanks a lot for the details. /var/lib/cyrus/annotations.db What are

Re: sync-server without deletes?

2010-09-14 Thread Shuvam Misra
With the new replication engine in 2.4, it will be possible - deleted messages still get replicated for a week - and if you set an explicit long expiry time on the replica (say, years!) then it wouldn't get cleaned up any earlier. That sounds like it's what we want, so I'll plan

De-duping attachments

2010-09-14 Thread Shuvam Misra
How difficult or easy would it be to modify Cyrus to strip all attachments from emails and store them separately in files? In the message file, replace the attachment with a special tag which will point to the attachment file. Whenever the message is fetched for any reason, the original

Re: De-duping attachments

2010-09-14 Thread Shuvam Misra
Dear Rob, I had reservations about some of these things too. :( In particular, I was wondering about having to remember and recreate the exact transfer-encoding. If both of us forward the same attachment in two emails, and one encodes in quoted-printable, the other in base64, Cyrus had better be

Re: De-duping attachments

2010-09-14 Thread Shuvam Misra
Dear Bron, http://www.newegg.com/Product/Product.aspx?Item=N82E16822148413 2TB - US $109. Don't want to nit-pick here, but the effective price we pay is about ten times this. To set up a mail server with a few TB of disk space, we usually land up deploying a separate chassis with RAID

Re: Another basic question about Cyrus replication

2010-09-13 Thread Shuvam Misra
One more question about sync-server and sync-client. Suppose I have two active servers, A and B, which contain completely disjoint sets of mailboxes. Can both replicate simultaneously to a replica server C? I will run sync-server only on C, and sync-client on A and B, pointing them both

Re: Store documents in IMAP folders

2010-09-12 Thread Shuvam Misra
I am wondering if it would be possible to use mailboxes as document repositories (in addition to emails) and how it could be done. Should the document be wrapped in a MIME envelop before being uploaded ? Any software that would actually do that? We too have felt that IMAP would be a good

Another basic question about Cyrus replication

2010-09-12 Thread Shuvam Misra
Dear all, One more question about sync-server and sync-client. Suppose I have two active servers, A and B, which contain completely disjoint sets of mailboxes. Can both replicate simultaneously to a replica server C? I will run sync-server only on C, and sync-client on A and B, pointing them both

Re: A beginner question about Murder

2010-09-09 Thread Shuvam Misra
On Wed, Sep 08, 2010 at 01:41:26PM -0700, Andrew Morgan wrote: Unfortunately, I've never setup a unified Murder, so I don't fully understand what the advantages and disadvantages of it compared to a traditional Murder. Maybe someone else can jump in here with their experiences. And

Re: A beginner question about Murder

2010-09-09 Thread Shuvam Misra
Dear Andrew, In a traditional Cyrus Murder (not a unified Murder), there are 3 roles: 1. backends - these store email 2. frontends - these proxy incoming connections to the correct backend 3. mupdate master - maintains the list of mailboxes in the Murder There can only be 1 mupdate

Re: Full Murder stack on one host

2010-09-09 Thread Shuvam Misra
Why is this a problem with current Cyrus Murder setups? What internal details prevent a front-end server, mupdate server, and back-end server from coexisting on the same physical system? Is it that there's no facility in the back-end server feature to make it listen only on

Re: A beginner question about Murder

2010-09-09 Thread Shuvam Misra
Does this mean that all cyradm-type admin connections must connect to one or other of the back-end servers? Can I get admin tasks done by connecting to one of the front-end servers too? Most admin tasks are proxied by the frontends. If defaultserver or serverlist aren't set, create must

Re: A beginner question about Murder

2010-09-09 Thread Shuvam Misra
What internal details prevent a front-end server, mupdate server, and back-end server from coexisting on the same physical system? Is it that there's no facility in the back-end server feature to make it listen only on localhost:imap and not on *:imap? No. Mostly it's a format difference

Re: A beginner question about Murder

2010-09-09 Thread Shuvam Misra
Yes, this. Absolutely. The replication code is pretty safe for multi-master in my branch already. At least for mailboxes. Sieve, Seen and Subs are somewhat trickier. I think the only really safe way is to keep deletion entries around and replicate those too so you can tell the difference

A beginner question about Murder

2010-09-08 Thread Shuvam Misra
Dear all, If I have, say, three IMAP servers each hosting a few thousand mailboxes, and I want to aggregate all of them for the IMAP client, I'll run Murder on one of the servers. 1. Can I run Murder on one of the back-end servers? If yes, it will act as an aggregator for incoming

Re: Basic question about Cyrus replication

2010-09-07 Thread Shuvam Misra
How do I prevent the replica server from listening on the imap port? Do I do this by not running imapd (from cyrus.conf)? If yes, then I guess the same needs to be done for POP3 and NNTP too, right? Correct. cyrus.conf's services section should contain syncserver, and ptloader, if you need

Basic question about Cyrus replication

2010-09-06 Thread Shuvam Misra
Hi all, Can you please give me some inputs about how Cyrus replication works? I've read the one page that comes with v2.3, and we've been using Cyrus (without replication) for a long time now. When I set up a master and a replica server, does the replica server listen on the IMAP port too, and

Re: Basic question about Cyrus replication

2010-09-06 Thread Shuvam Misra
When I set up a master and a replica server, does the replica server listen on the IMAP port too, and can it handle IMAP queries while it is receiving sync logs for rolling replication? No, the replica only listens on the sync server port. The replica should not listen on the IMAP port and

Need info on Cyrus v2.3 replication

2010-09-05 Thread Shuvam Misra
Guys, Can you please point me to any good online documentation on features and installation/configuration instructions for Cyrus mailbox replication (I believe this feature has been added in v2.3)? Whatever pointers I am getting from Google are to the old Cyrus IMAP Website at cmu.edu -- these

Re: Need info on Cyrus v2.3 replication

2010-09-05 Thread Shuvam Misra
Found a bunch of official HTML pages in the doc directory of a Cyrus distribution on one of our servers running v2.3.7 Cyrus. Thanks for the patience. Shuvam Guys, Can you please point me to any good online documentation on features and installation/configuration instructions for Cyrus