Re: Using xfer to migrate mailboxes to a new server

2010-02-18 Thread Per-Olov Sjöholm

On 15 feb 2010, at 15.59, Elver Loho wrote:

 On 15 February 2010 16:21, Kevin Kobb kk...@skylinecorp.com wrote:
 The last time we moved to new hardware, I used imapsync to migrate all
 the mailboxes to the new hardware. We moved from a different IMAP server
 to Cyrus, and this worked great. I don't know if this work as well now
 moving from Cyrus to Cyrus, as I don't think it would pick up quotas and
 believe xfer will.
 
 imapsync sounds like it could be the right tool for the job. Thanks!
 I'll check it out :)
 
 Best,
 Elver
 
 Cyrus Home Page: http://cyrusimap.web.cmu.edu/
 Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
 List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html


Hi

I did

1. create new mailboxes on target server and added quotas

2. The below
XXX.YYY.COM:/home # more /tmp/migrate-imap.sh 
./imapsync  --host1 192.168.3.5 --user1 per --passfile1 /root/passfile-per 
--authmech1 plain --host2 XXX.YYY.COM --user
2 per --passfile2 /root/passfile-per --authmech2 plain --delete2  --expunge2 
--syncinternaldates --subscribe --syncacls --ex
clude 'Other Users.*' --exclude 'Shared Folders.*'
./imapsync  --host1 192.168.3.5 --user1 john --passfile1 /root/passfile-john 
--authmech1 plain --host2 XXX.YYY.COM --user2 john 
--passfile2 /root/passfile-john --authmech2 plain --delete2  --expunge2 
--syncinternaldates --subscribe --syncacls --exclude 'Other
 Users.*' --exclude 'Shared Folders.*'
---cut the rest


It's resonalbe fast

The bad thing is that you need all passwords.

So this is not an ideal solution, but for sure it works great when you don't 
have all the time you would like to dig for a better solution. I did this for 
only 20-30 persons and could therefor live with the password issue. A few of 
them had mailboxes of 3-6 GB each. 

Hope it could be of any help


/Per-Olov
--
GPG keyID: 5231C0C4
GPG fingerprint: B232 3E1A F5AB 5E10 7561 6739 766E D29D 5231 C0C4
GPG key: http://pgp.mit.edu:11371/pks/lookup?op=getsearch=0x766ED29D5231C0C4


Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

How to migrate mail to a new server

2010-02-18 Thread J. Roeleveld
Hi All,

I have been reading the thread about using xfer to migrate to a different 
server or to use imapsync instead.
Both methods appear to require some additional work and I would prefer to 
simply copy the data across instead of using synchronization tools on live 
servers. (It's not a requirement to keep the email accessible during the 
transfer)

I have around 21GB of mail to copy across and was wondering how best to do 
this.
The mail server can (and most likely will) be stopped during the transfer.

Will it be sufficient to simply copy the /var/imap and /var/spool/imap 
folders across?
The contents of /var/spool/imap are all the emails and the db-files and such 
appear to be in /var/imap

Or will this cause problems and is there a better/recommended method of doing 
this?

Many thanks in advance,

Joost Roeleveld

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html


Re: Using xfer to migrate mailboxes to a new server

2010-02-18 Thread Reinaldo de Carvalho
On Thu, Feb 18, 2010 at 7:40 AM, Per-Olov Sjöholm p...@incedo.org wrote:
 It's resonalbe fast
 The bad thing is that you need all passwords.

No, use cyrus global admin.

 So this is not an ideal solution, but for sure it works great when you don't
 have all the time you would like to dig for a better solution. I did this
 for only 20-30 persons and could therefor live with the password issue. A
 few of them had mailboxes of 3-6 GB each.
 Hope it could be of any help


-- 
Reinaldo de Carvalho
http://korreio.sf.net
http://python-cyrus.sf.net

Don't try to adapt the software to the way you work, but rather
yourself to the way the software works (myself)

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html


Re: Using xfer to migrate mailboxes to a new server

2010-02-18 Thread J. Roeleveld
On Thursday 18 February 2010 12:31:02 Reinaldo de Carvalho wrote:
 On Thu, Feb 18, 2010 at 7:40 AM, Per-Olov Sjöholm p...@incedo.org wrote:
  It's resonalbe fast
  The bad thing is that you need all passwords.
 
 No, use cyrus global admin.

Won't that kill the seen database?

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html


Re: mailbox migration/repair question

2010-02-18 Thread Joe Vieira
Does any one know which cyrus file contains this?  Can I copy it over 
without the others?

Joe

Andrew Morgan wrote:
 On Wed, 17 Feb 2010, Joe Vieira wrote:

   
 Hi,

 So, We have an imap volume (ext3) that doesn't seem to come back up 
 clean after a few crashes.  fsck is a mess.  Shows clean; mount it; 30 
 minutes later it's throwing errors and needs to be fsck'd again.  (any 
 ideas about that i'd love to hear)

 So I am considering in the interest of getting these students back using 
 email, setting up a new volume, mount it as the same imap partition and 
 creating shell mailboxes for all the users on it, basically an empty 
 mailbox that I can let them log into, send and receive new mail, etc. 
 I was thinking about doing this by making the simple directory structure 
 for all the users then doing a reconstruct to make the cyrus.* files. 
 The plan would then be to restore from backup (or the corrupt drive in 
 read only). My concern however is that the UID of the messages will get 
 duplicated.  Basically does this idea make ANY sense to you guys?  Will 
 it work?  Other ideas?
 

 One of those cyrus.* files should contain the max UID - I can't remember 
 which.  If you can restore/copy that across, it should (I think) number 
 any new messages starting from there.  Then you can restore/copy the 
 original messages across later.

 I've never tried this before, so I may be getting it wrong.  Hopefully 
 someone with more in-depth knowledge of Cyrus can comment.

   Andy
   


Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html


Re: mailbox migration/repair question

2010-02-18 Thread Joe Vieira
For a totally less seamless solution:

What if I created the partition and created the empty mailboxes then 
moved the restored mail back in as restored-foldername This would ruin 
the seen.db, but...at least it would all be there in a pretty logical 
location.

Please weight in on this and tell me if you think it's feasible/advisable.

Thanks you SO MUCH!

Joe

Joe Vieira wrote:
 Does any one know which cyrus file contains this?  Can I copy it over 
 without the others?

 Joe

 Andrew Morgan wrote:
   
 On Wed, 17 Feb 2010, Joe Vieira wrote:

   
 
 Hi,

 So, We have an imap volume (ext3) that doesn't seem to come back up 
 clean after a few crashes.  fsck is a mess.  Shows clean; mount it; 30 
 minutes later it's throwing errors and needs to be fsck'd again.  (any 
 ideas about that i'd love to hear)

 So I am considering in the interest of getting these students back using 
 email, setting up a new volume, mount it as the same imap partition and 
 creating shell mailboxes for all the users on it, basically an empty 
 mailbox that I can let them log into, send and receive new mail, etc. 
 I was thinking about doing this by making the simple directory structure 
 for all the users then doing a reconstruct to make the cyrus.* files. 
 The plan would then be to restore from backup (or the corrupt drive in 
 read only). My concern however is that the UID of the messages will get 
 duplicated.  Basically does this idea make ANY sense to you guys?  Will 
 it work?  Other ideas?
 
   
 One of those cyrus.* files should contain the max UID - I can't remember 
 which.  If you can restore/copy that across, it should (I think) number 
 any new messages starting from there.  Then you can restore/copy the 
 original messages across later.

 I've never tried this before, so I may be getting it wrong.  Hopefully 
 someone with more in-depth knowledge of Cyrus can comment.

  Andy
   
 

 
 Cyrus Home Page: http://cyrusimap.web.cmu.edu/
 Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
 List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
   


Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html


Re: mailbox migration/repair question

2010-02-18 Thread Joe Vieira
So, if I create the directory structure for cyrus... 
spool/s/user/systemsmonitors  then reconstruct user.systemsmonitors  and 
try to deliver a message i get a mailbox does not exist error...

cat 4EA39BD0EEF |  /usr/lib/cyrus-imapd/deliver -e -r jvie...@clarku.edu 
user.systemsmonitors
user.systemsmonitors: Mailbox does not exist

Ideas?


Joe Vieira wrote:
 For a totally less seamless solution:

 What if I created the partition and created the empty mailboxes then 
 moved the restored mail back in as restored-foldername This would ruin 
 the seen.db, but...at least it would all be there in a pretty logical 
 location.

 Please weight in on this and tell me if you think it's feasible/advisable.

 Thanks you SO MUCH!

 Joe

 Joe Vieira wrote:
   
 Does any one know which cyrus file contains this?  Can I copy it over 
 without the others?

 Joe

 Andrew Morgan wrote:
   
 
 On Wed, 17 Feb 2010, Joe Vieira wrote:

   
 
   
 Hi,

 So, We have an imap volume (ext3) that doesn't seem to come back up 
 clean after a few crashes.  fsck is a mess.  Shows clean; mount it; 30 
 minutes later it's throwing errors and needs to be fsck'd again.  (any 
 ideas about that i'd love to hear)

 So I am considering in the interest of getting these students back using 
 email, setting up a new volume, mount it as the same imap partition and 
 creating shell mailboxes for all the users on it, basically an empty 
 mailbox that I can let them log into, send and receive new mail, etc. 
 I was thinking about doing this by making the simple directory structure 
 for all the users then doing a reconstruct to make the cyrus.* files. 
 The plan would then be to restore from backup (or the corrupt drive in 
 read only). My concern however is that the UID of the messages will get 
 duplicated.  Basically does this idea make ANY sense to you guys?  Will 
 it work?  Other ideas?
 
   
 
 One of those cyrus.* files should contain the max UID - I can't remember 
 which.  If you can restore/copy that across, it should (I think) number 
 any new messages starting from there.  Then you can restore/copy the 
 original messages across later.

 I've never tried this before, so I may be getting it wrong.  Hopefully 
 someone with more in-depth knowledge of Cyrus can comment.

 Andy
   
 
   
 
 Cyrus Home Page: http://cyrusimap.web.cmu.edu/
 Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
 List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
   
 

 
 Cyrus Home Page: http://cyrusimap.web.cmu.edu/
 Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
 List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
   


Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html


Re: mailbox migration/repair question

2010-02-18 Thread Andrew Morgan
On Thu, 18 Feb 2010, Joe Vieira wrote:

 For a totally less seamless solution:

What if I created the partition and created the empty mailboxes then
 moved the restored mail back in as restored-foldername This would ruin
 the seen.db, but...at least it would all be there in a pretty logical
 location.

 Please weight in on this and tell me if you think it's feasible/advisable.

That was the other idea I had, but obviously you'll lose all your seen 
information.  This is the same method I use when restoring messages for a 
user (restore in a new sub-folder).

Andy

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html


Re: mailbox migration/repair question

2010-02-18 Thread Andrew Morgan
On Thu, 18 Feb 2010, Joe Vieira wrote:

 So, if I create the directory structure for cyrus...
 spool/s/user/systemsmonitors  then reconstruct user.systemsmonitors  and
 try to deliver a message i get a mailbox does not exist error...

 cat 4EA39BD0EEF |  /usr/lib/cyrus-imapd/deliver -e -r jvie...@clarku.edu
 user.systemsmonitors
 user.systemsmonitors: Mailbox does not exist

 Ideas?

Did the mailbox appear in the mailboxes database?  Run ctl_mboxlist -d 
to see a plaintext dump of it.  You may have to create the mailbox using 
cm user.systemsmonitors first.

Andy

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html


Re: Using xfer to migrate mailboxes to a new server

2010-02-18 Thread nodens2099
On 18/02/2010 13:08, J. Roeleveld wrote:
 On Thursday 18 February 2010 12:31:02 Reinaldo de Carvalho wrote:
 On Thu, Feb 18, 2010 at 7:40 AM, Per-Olov Sjöholmp...@incedo.org  wrote:
 It's resonalbe fast
 The bad thing is that you need all passwords.

 No, use cyrus global admin.

 Won't that kill the seen database?

Only if you don't use the cyrus global admin with proxy authentication, 
to impersonate the user without knowing his password (--user1 / --user2 
vs --authuser1 / --authuser2)

Cheers,

-- 
Clément Hermann (nodens)

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html


Re: mailbox migration/repair question

2010-02-18 Thread Joe Vieira
Andrew Morgan wrote:
 On Thu, 18 Feb 2010, Joe Vieira wrote:

   
 So, if I create the directory structure for cyrus...
 spool/s/user/systemsmonitors  then reconstruct user.systemsmonitors  and
 try to deliver a message i get a mailbox does not exist error...

 cat 4EA39BD0EEF |  /usr/lib/cyrus-imapd/deliver -e -r jvie...@clarku.edu
 user.systemsmonitors
 user.systemsmonitors: Mailbox does not exist

 Ideas?
 

 Did the mailbox appear in the mailboxes database?  Run ctl_mboxlist -d 
 to see a plaintext dump of it.  You may have to create the mailbox using 
 cm user.systemsmonitors first.

   Andy
   
Yea, I dunno I apparently was being stupid with deliver and using it 
wrong... because it seems fine now =)

Joe

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html


Re: Setting TCP keepalive for Cyrus daemons

2010-02-18 Thread Gary Mills
On Sun, Feb 14, 2010 at 09:30:49AM -0600, Gary Mills wrote:
 
 I just noticed something else when I went to apply the patch.  I would
 have added the options to cyrus.conf so a typical entry would change
 from:
 
   imap  cmd=imapd listen=imap proto=tcp4 prefork=0 maxchild=6000
 
 to:
 
   imap  cmd=imapd listen=imap proto=tcp4 tcp_keepalive prefork=0 
 maxchild=6000
 
 That way you could have a different keepalive setting for each service.
 You've designed it so these settings go into imapd.conf .  Is that going
 to work the same way?

I haven't seen a response on this question.  Is it better to set TCP
options for Cyrus daemons in /etc/imapd.conf or /etc/cyrus.conf?  I'm
willing to test this facility, but I'd like to know where the settings
should be made first.

-- 
-Gary Mills--Unix Group--Computer and Network Services-

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html


Any interest to implement RFC4978 (IMAP COMPRESS)?

2010-02-18 Thread Pascal Gienger
RFC 4978 [1] defines an IMAP COMPRESS command to compress IMAP data 
communication.

Is there any interest to implement this extension in the cyrus imap server?

For low bandwith connections this could be useful but I don't know if 
that's a typical case nowadays. Together with the IMAP IDLE command it 
should be fine for mobile devices...

[1] http://tools.ietf.org/html/rfc4978

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html


Re: Any interest to implement RFC4978 (IMAP COMPRESS)?

2010-02-18 Thread Kenneth Marshall
On Thu, Feb 18, 2010 at 08:41:07PM +0100, Pascal Gienger wrote:
 RFC 4978 [1] defines an IMAP COMPRESS command to compress IMAP data 
 communication.
 
 Is there any interest to implement this extension in the cyrus imap server?
 
 For low bandwith connections this could be useful but I don't know if 
 that's a typical case nowadays. Together with the IMAP IDLE command it 
 should be fine for mobile devices...
 
 [1] http://tools.ietf.org/html/rfc4978

I thought that this was supported in 2.3.16.

Ken

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html


Re: Any interest to implement RFC4978 (IMAP COMPRESS)?

2010-02-18 Thread Rob Mueller

 For low bandwith connections this could be useful but I don't know if
 that's a typical case nowadays. Together with the IMAP IDLE command it
 should be fine for mobile devices...

 [1] http://tools.ietf.org/html/rfc4978

 I thought that this was supported in 2.3.16.

It's definitely in CVS, not sure what released version it's in (check the 
changelog?). We use it at FastMail.

http://blog.fastmail.fm/2009/05/01/help-test-proxy-to-improve-imap-performance/

Rob


Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html


Re: Any interest to implement RFC4978 (IMAP COMPRESS)?

2010-02-18 Thread Kenneth Marshall
On Fri, Feb 19, 2010 at 08:13:44AM +1100, Rob Mueller wrote:

 For low bandwith connections this could be useful but I don't know if
 that's a typical case nowadays. Together with the IMAP IDLE command it
 should be fine for mobile devices...

 [1] http://tools.ietf.org/html/rfc4978

 I thought that this was supported in 2.3.16.

 It's definitely in CVS, not sure what released version it's in (check the 
 changelog?). We use it at FastMail.

 http://blog.fastmail.fm/2009/05/01/help-test-proxy-to-improve-imap-performance/

 Rob

We are running 2.3.16 and it is definitely there. I think it was
in 2.3.15 but there were some bugs with Thunderbird that were
fixed in 2.3.16.

Regards,
Ken

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html