Re: Another basic question about Cyrus replication
One more question about sync-server and sync-client. Suppose I have two active servers, A and B, which contain completely disjoint sets of mailboxes. Can both replicate simultaneously to a replica server C? I will run sync-server only on C, and sync-client on A and B, pointing them both to C. Sort of - but it's not really recommended. Running a separate instance of Cyrus for each replica (with a different set of config files) would be more compatible with the way it's designed to work. Okay, got it, somewhat. Can you please elaborate what you mean by more compatible? If this is possible, I can maintain a slow backup server with large disks to maintain a backup of a dozen separate active Cyrus servers where the user mailboxes are distributed across this dozen. You'll find the replica still gets a lot of write IO. If your master copies are already getting a lot of writes, the replica is going to struggle. Yes, this occurred to me later. I realised that I may be able to work with a lower-powered CPU, but will find it difficult to work unless the replica server had high enough IOPS to handle all the disk I/O. :) thanks a lot, Shuvam Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: New Cyrus project site and bugzilla
Dave McMurtrie wrote: We're very interested in growing the Cyrus project and attracting new volunteers to contribute to the project, and that desire is at the core of why this migration is taking place. The biggest change is that we're trying to separate the environment from Carnegie Mellon University infrastructure as much as possible. Previously, contributions of any kind would end up requiring us to create a CMU computer account for a willing volunteer. We can now simply create local shell accounts as required. Almost the entire website has been created using MediaWiki software, so anyone who is willing to register for an account may update the website content. Wow - this looks really good :) The main criticism I have from a developer point of view is, well, CVS. Enough said. Please please can we have an official git mirror? It makes maintaining out-of-tree patches so much easier in the long run, and therefore much more likely that we can pass the patches back upstream. (On a separate note, if I go to Downloads - Getting Started and click on the AnonymousCVS wiki link then I get redirected back to the front page rather than to a page giving information on how to access CVS) ATB, Mark. -- Mark Cave-Ayland - Senior Technical Architect PostgreSQL - PostGIS Sirius Corporation plc - control through freedom http://www.siriusit.co.uk t: +44 870 608 0063 Sirius Labs: http://www.siriusit.co.uk/labs Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: New Cyrus project site and bugzilla
On Mon, Sep 13, 2010 at 09:09:53AM +0100, Mark Cave-Ayland wrote: Dave McMurtrie wrote: We're very interested in growing the Cyrus project and attracting new volunteers to contribute to the project, and that desire is at the core of why this migration is taking place. The biggest change is that we're trying to separate the environment from Carnegie Mellon University infrastructure as much as possible. Previously, contributions of any kind would end up requiring us to create a CMU computer account for a willing volunteer. We can now simply create local shell accounts as required. Almost the entire website has been created using MediaWiki software, so anyone who is willing to register for an account may update the website content. Wow - this looks really good :) The main criticism I have from a developer point of view is, well, CVS. Enough said. Please please can we have an official git mirror? It makes maintaining out-of-tree patches so much easier in the long run, and therefore much more likely that we can pass the patches back upstream. We're working on it! I'm hoping to chat with Jeroen from Kolab about it again tonight. We've got a partial merge into git - but we just want to make sure all the tags and authors and stuff are imported properly before cutting over. And to give people enough warning to change :) (On a separate note, if I go to Downloads - Getting Started and click on the AnonymousCVS wiki link then I get redirected back to the front page rather than to a page giving information on how to access CVS) Punt to Dave :) P.S. I'm getting going with Documentation now! I've rewritten the mailbox-format.html internal doc and added a namelocks.html internal doc. Next step is documenting the APIs at the new layer up (mailbox API) and the one above that (index API). And the replication protocol, and the sequence sets, buffers, etc. And the new reconstruct function, and the at-startup cvt_cyrusdb. I'm still looking for an expert BDB person to help me debug how Cyrus uses BDB :) Bron. Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Running multiple instances of cyrus for clustering
I am thinking of running mutiple instances of cyrus on a single machine with different sets of mailboxes. The Idea is that I would have two cyrus imap servers running on different machines and in case of any failure both instances will be run from the same machine ( obviously at a lower performance ) Is this a good idea ? Thanks Ram Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: New Cyrus project site and bugzilla
Bron Gondwana wrote: The main criticism I have from a developer point of view is, well, CVS. Enough said. Please please can we have an official git mirror? It makes maintaining out-of-tree patches so much easier in the long run, and therefore much more likely that we can pass the patches back upstream. We're working on it! I'm hoping to chat with Jeroen from Kolab about it again tonight. We've got a partial merge into git - but we just want to make sure all the tags and authors and stuff are imported properly before cutting over. And to give people enough warning to change :) Excellent news! FWIW the PostgreSQL team have been trying to switch to git for the past month, and in the process have involved the cvs2git maintainers and had some fixes committed over the past few weeks to improve the migration process (note that they have also suffered from having to hand-tweak the repository to fix various bugs in CVS). The thread about the entire process is very long, but for those interested the latest summary is here: http://archives.postgresql.org/pgsql-hackers/2010-09/msg00636.php. HTH, Mark. -- Mark Cave-Ayland - Senior Technical Architect PostgreSQL - PostGIS Sirius Corporation plc - control through freedom http://www.siriusit.co.uk t: +44 870 608 0063 Sirius Labs: http://www.siriusit.co.uk/labs Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Running multiple instances of cyrus for clustering
Hi, Quoting Ram r...@netcore.co.in: I am thinking of running mutiple instances of cyrus on a single machine with different sets of mailboxes. The Idea is that I would have two cyrus imap servers running on different machines and in case of any failure both instances will be run from the same machine ( obviously at a lower performance ) Is this a good idea ? IMHO yes, we want to run this kind of setup in produktion soon. We have choosen to use Cyrus Murder to ease failover. Each server will run 1 frontend, 1 backend and 1 replication server in case of failure of one server the corresponding replic will become backend. If the failed server is back onlie we will use replication to bring the original backend up to date. We will use ClusterIP for loadbalancing the Frontend connections. M.MengeTel.: (49) 7071/29-70316 Universität Tübingen Fax.: (49) 7071/29-5912 Zentrum für Datenverarbeitung mail: michael.me...@zdv.uni-tuebingen.de Wächterstraße 76 72074 Tübingen smime.p7s Description: S/MIME Signatur Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: CVS to GIT (was: New Cyrus project site and bugzilla)
Mark Cave-Ayland wrote: Bron Gondwana wrote: The main criticism I have from a developer point of view is, well, CVS. Enough said. Please please can we have an official git mirror? It makes maintaining out-of-tree patches so much easier in the long run, and therefore much more likely that we can pass the patches back upstream. We're working on it! I'm hoping to chat with Jeroen from Kolab about it again tonight. We've got a partial merge into git - but we just want to make sure all the tags and authors and stuff are imported properly before cutting over. And to give people enough warning to change :) Excellent news! FWIW the PostgreSQL team have been trying to switch to git for the past month, and in the process have involved the cvs2git maintainers and had some fixes committed over the past few weeks to improve the migration process (note that they have also suffered from having to hand-tweak the repository to fix various bugs in CVS). The thread about the entire process is very long, but for those interested the latest summary is here: http://archives.postgresql.org/pgsql-hackers/2010-09/msg00636.php. We have a working sample, with a documented procedure, to move three CVS modules (cmulocal, cyrus and sieve) into one GIT repository: http://www.cyrusimap.org/mediawiki/index.php/Drafts/CVS_to_GIT_Conversion#Stab_.232 There some about branch and tag conversions as well. You can find the result (which is preliminary!!) at http://git.kolabsys.com/cyrus-imapd.git. I'll be working with Dave to get this setup over on cyrusimap.org as soon as possible as well, but meanwhile, feedback is more then welcome! Kind regards, -- Jeroen van Meeuwen Senior Engineer, Kolab Systems AG e: vanmeeu...@kolabsys.com t: +316 42 801 403 w: http://www.kolabsys.com pgp: 9342 BF08 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Draft: Bugzilla Work Flow
Jeroen van Meeuwen (Kolab Systems) wrote: Hello there, Nudging ;-) CC:'ing the info list as well. I'm working on a documented Bugzilla work flow, in an attempt to streamline how we all work with it and what the average consumer may or may not expect. To allow some early feedback, I'm putting the page on the list now as opposed to when I feel like I'm done documenting everything in full ;-) http://www.cyrusimap.org/mediawiki/index.php/User:Jeroen_van_Meeuwen/Drafts/Bugzilla_Work_Flow Please note that these would, in any case, be guidelines, not law. There's no intention to make anyone's life any more difficult ;-) The attempt is to document what mere mortals like myself, and those people that are on the reporting end of bugs might expect, and to allow new contributors like myself to read up on what is an agreeable approach to handling Bugzilla issues. -- Jeroen van Meeuwen Senior Engineer, Kolab Systems AG e: vanmeeu...@kolabsys.com t: +316 42 801 403 w: http://www.kolabsys.com pgp: 9342 BF08 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Draft: Bugzilla Work Flow
On Mon, Sep 13, 2010 at 12:19:58PM +0200, Jeroen van Meeuwen (Kolab Systems) wrote: Jeroen van Meeuwen (Kolab Systems) wrote: Hello there, Nudging ;-) CC:'ing the info list as well. Sorry - didn't get a change to respond to this before! I'm working on a documented Bugzilla work flow, in an attempt to streamline how we all work with it and what the average consumer may or may not expect. To allow some early feedback, I'm putting the page on the list now as opposed to when I feel like I'm done documenting everything in full ;-) http://www.cyrusimap.org/mediawiki/index.php/User:Jeroen_van_Meeuwen/Drafts/Bugzilla_Work_Flow Please note that these would, in any case, be guidelines, not law. There's no intention to make anyone's life any more difficult ;-) The attempt is to document what mere mortals like myself, and those people that are on the reporting end of bugs might expect, and to allow new contributors like myself to read up on what is an agreeable approach to handling Bugzilla issues. That looks really good actually. I guess the side question is Is Bugzilla the ideal tool for this? I've seen setups where commit messages in the change management tool can be tied directly to tickets. It's probably not essential though - if we have a process and we all know how to follow it. Speaking of which - the auto bugzilla reminder emails are great :) Occasional annoyances that go away when you do the right thing are very useful. Bron. Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Draft: Bugzilla Work Flow
Bron Gondwana wrote: That looks really good actually. I guess the side question is Is Bugzilla the ideal tool for this? I've seen setups where commit messages in the change management tool can be tied directly to tickets. It's probably not essential though - if we have a process and we all know how to follow it. There's git-bugzilla for this purpose. I haven't actually worked with it though. There's also a python-bugzilla CLI client, which we could (possibly) use as a commit hook to update any tickets referred to in the commit message. Speaking of which - the auto bugzilla reminder emails are great :) Occasional annoyances that go away when you do the right thing are very useful. Yeah, so is the automagically put someone in CC: on bug update, having someone as a QA contact, or workflow enhancements (somebody patches some bug, closes the bug and release engineering looks to forward/backporting it). Kind regards, -- Jeroen van Meeuwen Senior Engineer, Kolab Systems AG e: vanmeeu...@kolabsys.com t: +316 42 801 403 w: http://www.kolabsys.com pgp: 9342 BF08 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: New Cyrus project site and bugzilla
On 09/13/2010 04:09 AM, Mark Cave-Ayland wrote: (On a separate note, if I go to Downloads - Getting Started and click on the AnonymousCVS wiki link then I get redirected back to the front page rather than to a page giving information on how to access CVS) Thanks for pointing this out, Mark. I made that link more useful. Dave -- Dave McMurtrie, SPE Email Systems Team Leader Carnegie Mellon University, Computing Services Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: CVS to GIT
Jeroen van Meeuwen (Kolab Systems) wrote: We have a working sample, with a documented procedure, to move three CVS modules (cmulocal, cyrus and sieve) into one GIT repository: http://www.cyrusimap.org/mediawiki/index.php/Drafts/CVS_to_GIT_Conversion#Stab_.232 There some about branch and tag conversions as well. You can find the result (which is preliminary!!) at http://git.kolabsys.com/cyrus-imapd.git. I'll be working with Dave to get this setup over on cyrusimap.org as soon as possible as well, but meanwhile, feedback is more then welcome! Kind regards, Oooh very nice. It seems to be a common issue that projects have to tweak their CVS repositories by hand to get a reasonable conversion to git. I'll try and take a closer look when I get a spare moment. My other point, of course, was that since the PostgreSQL guys worked to fix a couple of bugs in cvs2git over the past couple of weeks, you may get a better result if you grab the tip version of cvs2git and re-run the conversion. ATB, Mark. -- Mark Cave-Ayland - Senior Technical Architect PostgreSQL - PostGIS Sirius Corporation plc - control through freedom http://www.siriusit.co.uk t: +44 870 608 0063 Sirius Labs: http://www.siriusit.co.uk/labs Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: New Cyrus project site and bugzilla
Dave McMurtrie wrote: (On a separate note, if I go to Downloads - Getting Started and click on the AnonymousCVS wiki link then I get redirected back to the front page rather than to a page giving information on how to access CVS) Thanks for pointing this out, Mark. I made that link more useful. Dave Thanks Dave, that looks fixed now. ATB, Mark. -- Mark Cave-Ayland - Senior Technical Architect PostgreSQL - PostGIS Sirius Corporation plc - control through freedom http://www.siriusit.co.uk t: +44 870 608 0063 Sirius Labs: http://www.siriusit.co.uk/labs Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: CVS to GIT
Mark Cave-Ayland wrote: Oooh very nice. It seems to be a common issue that projects have to tweak their CVS repositories by hand to get a reasonable conversion to git. I'll try and take a closer look when I get a spare moment. Thanks! My other point, of course, was that since the PostgreSQL guys worked to fix a couple of bugs in cvs2git over the past couple of weeks, you may get a better result if you grab the tip version of cvs2git and re-run the conversion. If we were using cvs2git, sure! But we're not using cvs2git, we're using git- cvsimport ;-) Kind regards, -- Jeroen van Meeuwen Senior Engineer, Kolab Systems AG e: vanmeeu...@kolabsys.com t: +316 42 801 403 w: http://www.kolabsys.com pgp: 9342 BF08 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: CVS to GIT
On Mon, 13 Sep 2010 16:34:02 +0200, Jeroen van Meeuwen (Kolab Systems) vanmeeu...@kolabsys.com wrote: Mark Cave-Ayland wrote: My other point, of course, was that since the PostgreSQL guys worked to fix a couple of bugs in cvs2git over the past couple of weeks, you may get a better result if you grab the tip version of cvs2git and re-run the conversion. If we were using cvs2git, sure! But we're not using cvs2git, we're using git-cvsimport ;-) I would highly recommend cvs2git over git-cvsimport for reproducing an accurate history of the cvs repository. cvs2git doesn't do incremental imports like cvsimport, but I don't think that is needed in this situation. You may want to give it a try, just for a comparison anyway. -Brian Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Importing/moving an older cyrus message tree into a new system, without IMAP
I have an older system that crashed - cyrus version is a couple years or so old. I have 1000's of messages in the spool that I need to preserve. My question is about whether there's a way to import that huge tree of messages into a new cyrus installation without imap-to-imap connectivity? Thank you. Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Importing/moving an older cyrus message tree into a new system, without IMAP
On 13/09/10 18:14 -0400, Forrest Aldrich wrote: I have an older system that crashed - cyrus version is a couple years or so old. I have 1000's of messages in the spool that I need to preserve. My question is about whether there's a way to import that huge tree of messages into a new cyrus installation without imap-to-imap connectivity? If you're not concerned about your quota database, seen state, annotations, and subscription information, and assuming you've already regenerated your top level mailbox hierarchy, then you should be able to copy over the individual email files from each mailbox to the new server and perform a reconstruct on each mailbox (with the -r recursive option). If the new location is already live, then you'll need to be careful that you don't hit any filename collisions between the old server (e.g. email '123.') and the new server. You may also be able to copy over the primary database files (like your configdirectory/mailboxes.db), if your library version and cyrus versions match between the old and new servers. If not, you may need to use cvt_cyrusdb to convert the database from the old server to flat or skiplist and convert them back to their native format on the new server (berkeley db version mismatches are particularly a problem here). I would consider not copying over your deliver.db, tls_sessions.db, or your pts_cache.db if it exists. Those databases contain transient information that most consider to be non-critical. -- Dan White Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: CVS to GIT
Brian Awood wrote: On Mon, 13 Sep 2010 16:34:02 +0200, Jeroen van Meeuwen (Kolab Systems) vanmeeu...@kolabsys.com wrote: Mark Cave-Ayland wrote: My other point, of course, was that since the PostgreSQL guys worked to fix a couple of bugs in cvs2git over the past couple of weeks, you may get a better result if you grab the tip version of cvs2git and re-run the conversion. If we were using cvs2git, sure! But we're not using cvs2git, we're using git-cvsimport ;-) I would highly recommend cvs2git over git-cvsimport for reproducing an accurate history of the cvs repository. cvs2git doesn't do incremental imports like cvsimport, but I don't think that is needed in this situation. You may want to give it a try, just for a comparison anyway. I'm not sure what I am to understand from this. I've done thousands of conversions both with cvs2git as well as git-cvsimport. If you think there is a problem with git-cvsimport I may be unaware of, or there is a solution cvs2git has implemented that may or may not have been an actual problem with git-cvsimport, please point those out. Kind regards, -- Jeroen van Meeuwen Senior Engineer, Kolab Systems AG e: vanmeeu...@kolabsys.com t: +316 42 801 403 w: http://www.kolabsys.com pgp: 9342 BF08 Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
