Reconstruct on a live Cyrus system

2011-01-29 Thread Shuvam Misra
Guys, One basic question, which is not explicitly covered in the reconstruct manpage. Can I run a reconstruct on a user's mailbox when imapd is running and there's a risk that the user may connect to his mailbox? I need to know this for Cyrus v2.2 as well as later versions. Help, please? thanks,

Re: Shared mailboxes doc

2011-01-09 Thread Shuvam Misra
> On Fri, 2011-01-07 at 16:33 +, Andy Bennett wrote: > > Hi, > > > > Does anyone know the option that needs to be set (and how to set it) in > > > > order to do a "bulletin board". i.e. have a separate SEEN state for > > > each > > > > user? > > > Do the imapd.conf sharedseenstate option (

Re: GNU Mailutils

2010-12-20 Thread Shuvam Misra
> Mailutils has been around since dirt, but I've never heard of it being > used as a server. I do know that their IMAP support [client-side] is > crap which is why everyone uses/used c-client [derived from UW]. > > The manual for the imap4d services [which runs under xinted, yuck!] is > full of "

GNU Mailutils

2010-12-19 Thread Shuvam Misra
Hey guys, I know this is OT but any of you have any idea about this toolkit/system? I discovered it accidentally today --- didn't even know the GNU project had its own IMAP and POP3 servers in addition to client-side libraries. I wonder how good their IMAP daemon is, in terms of performance, admin

Re: ext3 / XFS

2010-11-16 Thread Shuvam Misra
> > This is depends on what filesystem you are useing, I have mailboxes with > > hundreds > > of thousands of messages in them on XFS and have no problems, but on ext3 I > > start seeing slowdowns with a bit over ten thousand messages. > > Was dir_index enabled on that ext3 filesystem? Prior t

Re: Does anyone allow unlimited or extremely large quotas?

2010-11-16 Thread Shuvam Misra
> On 16 Nov 2010, at 10:32, Joseph Brennan wrote: > > I wish we'd somehow financed a native Cyrus webmail interface, that is > > not using IMAP but built into Cyrus. I don't think users know how good > > Cyrus is because they look at it through a weak intermediary. > > I don't think a Cyrus-speci

Re: Does anyone allow unlimited or extremely large quotas?

2010-11-16 Thread Shuvam Misra
> On 16.11.2010 17:11, Shuvam Misra wrote: > > I guess Webmail is OT on a Cyrus mailing list, but can't help asking: any > > suggestions for > > improving Webmail performance? (Admission: we haven't yet tried imapproxy > > -- it appears to be a good piece o

Re: Does anyone allow unlimited or extremely large quotas?

2010-11-16 Thread Shuvam Misra
> Didn't Dave write up.imapproxy? It makes a huge difference for, e.g., > IMP & roundcube. Also, configuring client to not retrieve the LIST of > mailboxes during every transaction is a big win. Thanks a lot -- will definitely incorporate it into our setup. How does one configure the client not

Re: Does anyone allow unlimited or extremely large quotas?

2010-11-16 Thread Shuvam Misra
> > I think the issue you will encounter first is clients will start to fall > > down when folders exceed a 'reasonable' number of messages. Common IMAP > > clients I've seen start to exhibit severe performance issues beyond a > > few hundred thousand messages. > > As far as I'm aware (the helpde

Re: replication

2010-11-12 Thread Shuvam Misra
> Quoting Bron Gondwana : > > > > It's getting better, but it's still not 100% reliable to have > > master/master replication between two servers with interactions > > going to both sides. > > > > It SHOULD be safe now to have a single master/master setup with > > individual users on one side or th

Re: IMAP not seeing old mail present on filesystem

2010-10-04 Thread Shuvam Misra
Dear Patrick, > How would you do the IMAP append? Using a Perl::IMAP function? > > This isn't necessarily a concern for this list, but a few days ago I > upgraded a site from cyrus 2.1.16 to cyrus 2.3.16 by using imapsync to > transfer mail from the old server to the new one. This worked grea

Re: competition

2010-09-26 Thread Shuvam Misra
> > For situations where we need just random access, not sequential, can we > > use GDBM? Is that library better than Berkeley DB? > ^ > > G => GNU => GPL. Licencing issues I suspect. We're BSD licence, > not GPL. Yes, you're quite right, I just checked. Till your comment, I had assumed t

Re: competition

2010-09-25 Thread Shuvam Misra
> On 9/22/2010 10:20 PM, Shuvam Misra wrote: > > I was a strong advocate of bundling DB libraries, etc, with Cyrus. The > > points you've made here are very interesting. I didn't know many of these > > things. I'm re-thinking whether bundling is such a good ide

Re: competition

2010-09-22 Thread Shuvam Misra
> Kolab Systems is thinking of such SQL databases for integration > purposes, where the performance penalty now lies within having to use the > IMAP protocol to gain access to the underlying metadata (seen status, > message indexes) in distributed groupware environments where Cyrus > itself is not

Re: competition

2010-09-22 Thread Shuvam Misra
> > given the issues with BDB. Is it worth embedding a copy of > > BDB into the Cyrus distribution rather than using the OS one? I > > That way lies madness. > > BDB is one of those things where arcane blackmagic skills are needed to keep > it working on all arches. It uses scary crap to

Berkeley DB: private copy?

2010-09-22 Thread Shuvam Misra
> given the issues with BDB. Is it worth embedding a copy of > BDB into the Cyrus distribution rather than using the OS one? I > know it's generally considered bad taste, but it sure makes > keeping in sync easier! IMHO, yes, most certainly. Cyrus is a large and complex system, and its main

Is the statuscache.db patch part of Cyrus now?

2010-09-21 Thread Shuvam Misra
Is this patch: http://www.mail-archive.com/cyrus-de...@lists.andrew.cmu.edu/msg01194.html part of Cyrus now? We recently read a post from one friend who said he's using this patch as part of the optimisations for his large setup. regards, Shuvam Cyrus Home Page: http://www.cyrusimap.or

RE: A beginner question about Murder

2010-09-16 Thread Shuvam Misra
> So we use about ten common Intel based servers for BEs (sized to > support loss of 2/3 servers) and store data on NAS. Users are active > and filer is about tens of thousand nfsop/s at the busy hour. > > About 10% to 20% of the users connect at least one time a day, globally > 4 million connectio

RE: A beginner question about Murder

2010-09-16 Thread Shuvam Misra
Dear Michel, > we use a modified "traditional" murder, i.e. without murder daemon, > to host more than 2 million mailboxes (dozen million entries in mboxlist > with folders) Wow, that's some figure. Care to share some details with the list? What kind of hardware (both for servers and storage), wh

Re: De-duping attachments

2010-09-15 Thread Shuvam Misra
> Makes sense. There might be some size based logic here too - only > bother applying this on messages over 20k, and where the attachment > is at least 20k in size. Anything smaller than that is pretty > pointless. Yes, absolutely. Left to myself, I'd not have bothered with any attachment less t

Re: Importing/moving an older cyrus message tree into a new system, without IMAP

2010-09-15 Thread Shuvam Misra
> Annotations are defined in RFC 5257. > > They allow an admin to add metadata to a mailbox (or the server). The > cyradm utility sets annotations with its internal info, mboxcfg, and > setinfo commands. Okay, checked. Don't know where these things are used, other than expiry and sieve, but at le

Re: De-duping attachments

2010-09-15 Thread Shuvam Misra
The sparse file idea is brilliant! Never occurred to me. :) We'd have to store the reference-pointer in the message file, so we would omit the actual attachment but eat up perhaps 50 bytes to keep the reference to the file. Shuvam > 1. Completely rewrite the message file removing the attachment

Re: De-duping attachments

2010-09-15 Thread Shuvam Misra
Dear Bron, > So you save, what, 50%. Does that sound about right? Do you have > statistics on how much space you'd save with this theoretical > patch? No, and this is the first thing I want to do. I'm getting some simple utilities developed which will run all week (niced suitably) and extract a

Re: De-duping attachments

2010-09-14 Thread Shuvam Misra
Dear Bron, > http://www.newegg.com/Product/Product.aspx?Item=N82E16822148413 > > 2TB - US $109. Don't want to nit-pick here, but the effective price we pay is about ten times this. To set up a mail server with a few TB of disk space, we usually land up deploying a separate chassis with RAID cont

Re: De-duping attachments

2010-09-14 Thread Shuvam Misra
Dear Rob, I had reservations about some of these things too. :( In particular, I was wondering about having to remember and recreate the exact transfer-encoding. If both of us forward the same attachment in two emails, and one encodes in quoted-printable, the other in base64, Cyrus had better be a

De-duping attachments

2010-09-14 Thread Shuvam Misra
How difficult or easy would it be to modify Cyrus to strip all attachments from emails and store them separately in files? In the message file, replace the attachment with a special tag which will point to the attachment file. Whenever the message is fetched for any reason, the original MIME-encode

Re: sync-server without deletes?

2010-09-14 Thread Shuvam Misra
> > > With the new replication engine in 2.4, it will be possible - deleted > > > messages still get replicated for a week - and if you set an explicit > > > long expiry time on the replica (say, years!) then it wouldn't get > > > cleaned up any earlier. > > > > That sounds like it's what we want,

Re: Importing/moving an older cyrus message tree into a new system, without IMAP

2010-09-14 Thread Shuvam Misra
Dear Dan, > See the manpage for imapd.conf for possible formats, but for my 2.3.12 > installation, with configdirectory specified at /var/lib/cyrus (and no > customization to my *_db options), my database files are: Got it. Thanks a lot for the details. > /var/lib/cyrus/annotations.db What are

Re: Importing/moving an older cyrus message tree into a new system, without IMAP

2010-09-14 Thread Shuvam Misra
Dear Dan, > If you're not concerned about your quota database, seen state, annotations, > and subscription information, and assuming you've already regenerated your > top level mailbox hierarchy, then you should be able to copy over the > individual email files from each mailbox to the new server

Re: Importing/moving an older cyrus message tree into a new system, without IMAP

2010-09-14 Thread Shuvam Misra
> We did a migration some months back from an old Kolab v1 (cyrus v2.1) > system to a new Kolab v2.2 (cyrus v2.2) system. > > This was done by writing a script to > > - dump the ldap database (you might not have this) and load it on the new >system > - rsync the mailboxes from their locatio

Re: Another basic question about Cyrus replication

2010-09-13 Thread Shuvam Misra
> > One more question about sync-server and sync-client. Suppose I have two > > active servers, A and B, which contain completely disjoint sets of > > mailboxes. Can both replicate simultaneously to a replica server C? I > > will run sync-server only on C, and sync-client on A and B, pointing them

Re: Store documents in IMAP folders

2010-09-12 Thread Shuvam Misra
> I am wondering if it would be possible to use mailboxes as document > repositories (in addition to emails) and how it could be done. > Should the document be wrapped in a MIME envelop before being uploaded ? > Any software that would actually do that? We too have felt that IMAP would be a good

Another basic question about Cyrus replication

2010-09-12 Thread Shuvam Misra
Dear all, One more question about sync-server and sync-client. Suppose I have two active servers, A and B, which contain completely disjoint sets of mailboxes. Can both replicate simultaneously to a replica server C? I will run sync-server only on C, and sync-client on A and B, pointing them both

Re: A beginner question about Murder

2010-09-09 Thread Shuvam Misra
> Yes, this. Absolutely. The replication code is pretty safe for multi-master > in my branch already. At least for mailboxes. Sieve, Seen and Subs are > somewhat trickier. I think the only really safe way is to keep "deletion" > entries around and replicate those too so you can tell the differ

Re: A beginner question about Murder

2010-09-09 Thread Shuvam Misra
> >What internal > >details prevent a front-end server, mupdate server, and back-end > >server > >from coexisting on the same physical system? Is it that there's no > >facility in the back-end server feature to make it listen only on > >localhost:imap and not on *:imap? > > No. Mostly it's a forma

Re: A beginner question about Murder

2010-09-09 Thread Shuvam Misra
> >Does this mean that all cyradm-type admin connections must connect > >to one > >or other of the back-end servers? Can I get admin tasks done by > >connecting to one of the front-end servers too? > > Most admin tasks are proxied by the frontends. If defaultserver or > serverlist aren't set, crea

Re: Full Murder stack on one host

2010-09-09 Thread Shuvam Misra
> > Why is this a problem with current Cyrus Murder setups? What internal > > details prevent a front-end server, mupdate server, and back-end server > > from coexisting on the same physical system? Is it that there's no > > facility in the back-end server feature to make it listen only on > > loca

Re: A beginner question about Murder

2010-09-08 Thread Shuvam Misra
Dear Andrew, > In a "traditional" Cyrus Murder (not a "unified" Murder), there are > 3 roles: > > 1. backends - these store email > 2. frontends - these proxy incoming connections to the correct backend > 3. mupdate master - maintains the list of mailboxes in the Murder > > There can only be 1 m

Re: A beginner question about Murder

2010-09-08 Thread Shuvam Misra
> On Wed, Sep 08, 2010 at 01:41:26PM -0700, Andrew Morgan wrote: > > Unfortunately, I've never setup a "unified" Murder, so I don't fully > > understand what the advantages and disadvantages of it compared to a > > "traditional" Murder. Maybe someone else can jump in here with their > > experie

A beginner question about Murder

2010-09-08 Thread Shuvam Misra
Dear all, If I have, say, three IMAP servers each hosting a few thousand mailboxes, and I want to aggregate all of them for the IMAP client, I'll run Murder on one of the servers. 1. Can I run Murder on one of the back-end servers? If yes, it will act as an aggregator for incoming connection

Re: Basic question about Cyrus replication

2010-09-07 Thread Shuvam Misra
> >How do I prevent the replica server from listening on the imap port? Do > >I do this by not running imapd (from cyrus.conf)? If yes, then I guess > >the same needs to be done for POP3 and NNTP too, right? > > Correct. cyrus.conf's services section should contain syncserver, > and ptloader, if

Re: Basic question about Cyrus replication

2010-09-06 Thread Shuvam Misra
> >When I set up a master and a replica server, does the replica server > >listen on the IMAP port too, and can it handle IMAP queries while it is > >receiving sync logs for rolling replication? > > No, the replica only listens on the sync server port. The replica > should not listen on the IMAP

Basic question about Cyrus replication

2010-09-06 Thread Shuvam Misra
Hi all, Can you please give me some inputs about how Cyrus replication works? I've read the one page that comes with v2.3, and we've been using Cyrus (without replication) for a long time now. When I set up a master and a replica server, does the replica server listen on the IMAP port too, and ca

Re: Need info on Cyrus v2.3 replication

2010-09-05 Thread Shuvam Misra
Found a bunch of official HTML pages in the doc directory of a Cyrus distribution on one of our servers running v2.3.7 Cyrus. Thanks for the patience. Shuvam > Guys, > > Can you please point me to any good online documentation on features and > installation/configuration instructions for Cyrus m

Need info on Cyrus v2.3 replication

2010-09-05 Thread Shuvam Misra
Guys, Can you please point me to any good online documentation on features and installation/configuration instructions for Cyrus mailbox replication (I believe this feature has been added in v2.3)? Whatever pointers I am getting from Google are to the old Cyrus IMAP Website at cmu.edu -- these UR