Re: cyrus replication question

2006-12-14 Thread stas khromoy 
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

still battling with replication :)

i've noticed that for only one
of the users
i get the following error

Error from do_user(-l): bailing out!

in the list archives i found some one with a similar issue
(at least is sounded like it)

the advice was to to reconstruct the mailbox
but on which machine - master or replica ?

also i don't see that error in the logs
it comes as output of sync_client -l -v -u (user list)

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.5 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQCVAwUBRYFisdbpM15f9s9nAQLLKQP+LZKXIPDkTr+GPKWjNxZxQMU/jco9xNxU
aALb+Xsh04Fhu6IovI4rXLOHBc7m46kTI/2SHqhlTtdFcVxJYfp/sPFxSJncnPjd
ECDDfeYte1Yj+jUO12yzUze8YWIPEZnUEnAE+ia4TK97C9oP//r/XRRzCt9E1XqA
CNyoZxDQNzE=
=8hR2
-END PGP SIGNATURE-

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: cyrus replication question

2006-12-14 Thread stas khromoy 



still battling with replication :)

i've noticed that for only one
of the users
i get the following error

Error from do_user(-l): bailing out!

in the list archives i found some one with a similar issue
(at least is sounded like it)

the advice was to to reconstruct the mailbox
but on which machine - master or replica ?

also i don't see that error in the logs
it comes as output of sync_client -l -v -u (user list)


Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: cyrus replication question

2006-12-14 Thread Wesley Craig 

On 14 Dec 2006, at 09:41, stas khromoy wrote:

i've noticed that for only one
of the users
i get the following error

Error from do_user(-l): bailing out!

in the list archives i found some one with a similar issue
(at least is sounded like it)


That's the error that you'd get from sync_client.  Probably a more  
detailed error is in syslog, and even more detail is likely to be  
available from sync_server on the replica.



the advice was to to reconstruct the mailbox
but on which machine - master or replica ?


reconstruct on the master.  Probably I'd run sync_reset on the user  
in question on the replica as well.  Then sync_client -u again.


:wes


also i don't see that error in the logs
it comes as output of sync_client -l -v -u (user list)


The error in the logs is probably more detailed, something like  
server responded with NO etc.


:wes

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: cyrus replication question

2006-12-14 Thread stas khromoy 
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

thanks for all your help, wesley

i'll try this as soon as i can

there is one question that is till puzzling me thou

when it replicates, it ignores inactive folders
for some reason

i looked at the size of my imap/user folder on replica (1.9gb)
and size of the same folder on master (2.4gb)

i am sure other mailboxes have the same issue
since the size of imap/user on replica is smaller then on master

any ideas ? or is it normal
i checked all the logs i dont see anything that could hint on the
problem if there is one


by inactive folders i mean folders that u use to store emails
and don't touch often

thanks again



Wesley Craig wrote:
 On 14 Dec 2006, at 09:41, stas khromoy wrote:
 i've noticed that for only one
 of the users
 i get the following error

 Error from do_user(-l): bailing out!

 in the list archives i found some one with a similar issue
 (at least is sounded like it)
 
 That's the error that you'd get from sync_client.  Probably a more
 detailed error is in syslog, and even more detail is likely to be
 available from sync_server on the replica.
 
 the advice was to to reconstruct the mailbox
 but on which machine - master or replica ?
 
 reconstruct on the master.  Probably I'd run sync_reset on the user in
 question on the replica as well.  Then sync_client -u again.
 
 :wes
 
 also i don't see that error in the logs
 it comes as output of sync_client -l -v -u (user list)
 
 The error in the logs is probably more detailed, something like server
 responded with NO etc.
 
 :wes
 
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.5 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQCVAwUBRYGbatbpM15f9s9nAQIAVwP+N2cdqdlhnhFsJdKvhqez2SvZ1KDI/HcT
2AzIdLtRiLztLr+k0aOwtO6lUXFogA/iWULDlkDyqhaHfa0v0mXzDK9fb8tJexXS
e96AtNRjwb+sVrXKvRzcMsgtMMCdEij5MIC1rLm60lTNzi1T75WC0cWQ2BkKRDbs
HVkQ4zEPXiA=
=BG0M
-END PGP SIGNATURE-

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: cyrus replication question

2006-12-14 Thread Wesley Craig 

On 14 Dec 2006, at 13:43, stas khromoy wrote:

i looked at the size of my imap/user folder on replica (1.9gb)
and size of the same folder on master (2.4gb)


First, I'd try to fix the bailing out problem you're having.  Or,  
if you want to just get things closer to correct, invoke:


sync_client -v -l -u username

once for every user.  That way, when it encounters one bad user, it  
won't stop.  Of course, you may have many bad users, as well.  If  
you still have problems once all the bad users are corrected, then  
we can debug the size differences.


:wes

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: cyrus replication question

2006-12-13 Thread stas khromoy 
did as you advised

now i see folder fro all the users on replica machine
but the size of the /imap/users has not changed.

basically whatever is inactive
be it a full mailbox or an active mailbox's subfolder
is not being replicated.


Wesley Craig wrote:
 Like this:
 
 sync_client -l -v -u userA userB userC ...
 
 Most Unix commands function this way.
 
 :wes
 
 On 12 Dec 2006, at 21:56, [EMAIL PROTECTED] wrote:
 do thay have to be separated (user names i mean) by space or coma or
 something else ??
 
 Cyrus Home Page: http://cyrusimap.web.cmu.edu/
 Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
 List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
 

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: cyrus replication question

2006-12-12 Thread Dmitriy Kirhlarov 
On Mon, Dec 11, 2006 at 04:35:24PM -0500, [EMAIL PROTECTED] wrote:
 
 i started looking around
 for one i found that for whatever reason
 both sync and some other default process was using 2005/tcp port
 (in /etc/services )
 after taking care of that
 
 i kicked off on sync_client on master server
 (after logging in as cyrus)
 on master server i get
 can not connect to server 'replica.mydomain.com trying in 15 seconds.
 
 and in /var/log/messages the following comes up
 (still on master server)
 
 Dec 11 16:09:56 master_server sync_client[93038]: TLS client engine:
 cannot load CA data
 Dec 11 16:09:56 master_server sync_client[93038]: Doing a peer verify
 Dec 11 16:09:56 master_server sync_client[93038]: verify error:num=18:self
 signed certificate
 Dec 11 16:09:56 master_server sync_client[93038]: starttls: TLSv1 with
 cipher AES256-SHA (256/256 bits new) no authentication
 Dec 11 16:09:57 master_server sync_client[93038]: No worthy mechs found
 Dec 11 16:09:57 master_server sync_client[93038]: couldn't authenticate to
 backend server: no mechanism available
 -

Check 
tls_key_file:
tls_ca_file:
tls_cert_file:

in your imapd.conf

WBR
-- 
Dmitriy Kirhlarov
OILspace, 26 Leninskaya sloboda, bld. 2, 2nd floor, 115280 Moscow, Russia
P:+7 495 105 7247 ext.208 F:+7 495 105 7246 E:[EMAIL PROTECTED]
OILspace - The resource enriched - www.oilspace.com

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: cyrus replication question

2006-12-12 Thread Dmitriy Kirhlarov 
On Mon, Dec 11, 2006 at 04:36:38PM -0500, [EMAIL PROTECTED] wrote:
 
 another question :
 
 do i need to create mailboxes
 on the replica server or will they be created by the replication
 process ?

You don't need create mailboxes manually.

WBR
-- 
Dmitriy Kirhlarov
OILspace, 26 Leninskaya sloboda, bld. 2, 2nd floor, 115280 Moscow, Russia
P:+7 495 105 7247 ext.208 F:+7 495 105 7246 E:[EMAIL PROTECTED]
OILspace - The resource enriched - www.oilspace.com

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: cyrus replication question

2006-12-12 Thread stas 
i did as you've said

this is now the output of synctest

S: * STARTTLS
S: * OK mars.edpausa.com Cyrus sync server v2.3.7
Authentication failed. generic failure
Security strength factor: 0

/var/log/messages shows :

Dec 12 20:56:21 mars syncserver[489]: login: [192.168.0.201] cyrus
PLAIN+TLS User logged in
Dec 12 20:56:35 mars syncserver[313]: starttls: TLSv1 with cipher
AES256-SHA (256/256 bits new) no authentication
Dec 12 20:56:35 mars syncserver[313]: login: [192.168.0.201] cyrus
PLAIN+TLS User logged in
 

it seems that something did happen
it replicated all the active mailboxes
( there are a few innactive from the people who recently left and a
few test mailboxes)
and replicated only active subfolders of these boxes.

so out of 121GB only 58 GB have been replicated

i see in logs
things like this
Dec 12 20:56:45 mars syncserver[313]: Expunged 1 messages from user.stas

Dec 12 21:01:50 mars syncserver[313]: skiplist: recovered
/var/imap/user/e/edpa.seen.313 (2 records, 11608 bytes) in 0 seconds

so something is defetnly going on
but why is it replicating only active mailboxes/subfolders ??




Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: cyrus replication question

2006-12-12 Thread Wesley Craig 

On 12 Dec 2006, at 21:08, [EMAIL PROTECTED] wrote:

but why is it replicating only active mailboxes/subfolders ??


If you're enabling replication on an already in-production server,  
you'll probably want to run something like:


sync_client -l -v -u list of users

That will cause all existing mailboxes to be replicated.  Thereafter,  
you shouldn't need to intervene (much).


:wes

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: cyrus replication question

2006-12-12 Thread stas 
do thay have to be separated (user names i mean) by space or coma or
something else ??

i am getting these
Dec 12 21:48:34 mars syncserver[313]: can not unlink
/var/imap/user/l/lenny,.seen: No such file or directory
Dec 12 21:48:34 mars syncserver[313]: can not unlink
/var/imap/user/l/lenny,.mboxkey: No such file or directory

when i do name, name, name
or name name or name,name




 On 12 Dec 2006, at 21:08, [EMAIL PROTECTED] wrote:
 but why is it replicating only active mailboxes/subfolders ??

 If you're enabling replication on an already in-production server,
 you'll probably want to run something like:

   sync_client -l -v -u list of users

 That will cause all existing mailboxes to be replicated.  Thereafter,
 you shouldn't need to intervene (much).

 :wes




Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: cyrus replication question

2006-12-12 Thread Wesley Craig 

Like this:

sync_client -l -v -u userA userB userC ...

Most Unix commands function this way.

:wes

On 12 Dec 2006, at 21:56, [EMAIL PROTECTED] wrote:

do thay have to be separated (user names i mean) by space or coma or
something else ??


Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

cyrus replication question

2006-12-11 Thread stas 

while trying to setup cyrus replication
i am getting the following error
on the secondary server ( the one that is getting a copy of everything)

Dec  9 12:35:34 mars master[7701]: about to exec 
/usr/local/cyrus/bin/sync_server
Dec  9 12:35:34 mars syncserver[7701]: executed
Dec  9 12:35:34 mars syncserver[7701]: accepted connection
Dec  9 12:35:34 mars syncserver[7701]: cmdloop(): startup
Dec  9 12:35:34 mars master[7694]: process 7701 exited, status 75
Dec  9 12:35:34 mars master[7694]: service syncserver pid 7701 in BUSY
state: terminated abnormally

here are the settings from both imapd.conf and cyrus.conf
for both master and backup server.

master server
(from where everything is being copied)

cyrus.conf

under START section:
syncclientcmd=/usr/local/cyrus/bin/sync_client -r -t


imapd.conf

#replication configurations
sync_host: server.mydomain.com
sync_authname: cyrus
sync_passwd: password
sync_machineid: 1
sync_log: 1

secondary
(to where everything is bein copied)
cyrus.conf

SERVICES {
syncservercmd=sync_server listen=csync






Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: cyrus replication question

2006-12-11 Thread stas 

another question :

do i need to create mailboxes
on the replica server or will they be created by the replication
process ?


 while trying to setup cyrus replication
 i am getting the following error
 on the secondary server ( the one that is getting a copy of everything)

 Dec  9 12:35:34 mars master[7701]: about to exec
 /usr/local/cyrus/bin/sync_server
 Dec  9 12:35:34 mars syncserver[7701]: executed
 Dec  9 12:35:34 mars syncserver[7701]: accepted connection
 Dec  9 12:35:34 mars syncserver[7701]: cmdloop(): startup
 Dec  9 12:35:34 mars master[7694]: process 7701 exited, status 75
 Dec  9 12:35:34 mars master[7694]: service syncserver pid 7701 in BUSY
 state: terminated abnormally

 here are the settings from both imapd.conf and cyrus.conf
 for both master and backup server.

 master server
 (from where everything is being copied)

 cyrus.conf

 under START section:
 syncclientcmd=/usr/local/cyrus/bin/sync_client -r -t


 imapd.conf

 #replication configurations
 sync_host: server.mydomain.com
 sync_authname: cyrus
 sync_passwd: password
 sync_machineid: 1
 sync_log: 1

 secondary
 (to where everything is bein copied)
 cyrus.conf

 SERVICES {
 syncservercmd=sync_server listen=csync





 
 Cyrus Home Page: http://cyrusimap.web.cmu.edu/
 Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
 List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html




Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: cyrus replication question

2006-12-11 Thread stas 

i started looking around
for one i found that for whatever reason
both sync and some other default process was using 2005/tcp port
(in /etc/services )
after taking care of that

i kicked off on sync_client on master server
(after logging in as cyrus)
on master server i get
can not connect to server 'replica.mydomain.com trying in 15 seconds.

and in /var/log/messages the following comes up
(still on master server)

Dec 11 16:09:56 master_server sync_client[93038]: TLS client engine:
cannot load CA data
Dec 11 16:09:56 master_server sync_client[93038]: Doing a peer verify
Dec 11 16:09:56 master_server sync_client[93038]: verify error:num=18:self
signed certificate
Dec 11 16:09:56 master_server sync_client[93038]: starttls: TLSv1 with
cipher AES256-SHA (256/256 bits new) no authentication
Dec 11 16:09:57 master_server sync_client[93038]: No worthy mechs found
Dec 11 16:09:57 master_server sync_client[93038]: couldn't authenticate to
backend server: no mechanism available
-

/var/log/messeges on replica server
is not saying anything at this point
but /var/log/debug.log has the following :


Dec 11 15:56:33 replica_server syncserver[92364]: accepted connection
Dec 11 15:56:33 replica_server syncserver[92364]: cmdloop(): startup
Dec 11 15:56:33 replica_server syncserver[92364]: mystore: starting txn
2147483668
Dec 11 15:56:33 replica_server syncserver[92364]: mystore: committing txn
2147483668
Dec 11 15:56:49 replica_server syncserver[92364]: accepted connection
Dec 11 15:56:49 replica_server syncserver[92364]: cmdloop(): startup
Dec 11 15:56:49 replica_server syncserver[92364]: mydelete: starting txn
2147483669
Dec 11 15:56:49 replica_server syncserver[92364]: mydelete: committing txn
2147483669
Dec 11 15:56:49 replica_server syncserver[92364]: mystore: starting txn
2147483670
Dec 11 15:56:49 replica_server syncserver[92364]: mystore: committing txn
2147483670

 while trying to setup cyrus replication
 i am getting the following error
 on the secondary server ( the one that is getting a copy of everything)

 Dec  9 12:35:34 mars master[7701]: about to exec
 /usr/local/cyrus/bin/sync_server
 Dec  9 12:35:34 mars syncserver[7701]: executed
 Dec  9 12:35:34 mars syncserver[7701]: accepted connection
 Dec  9 12:35:34 mars syncserver[7701]: cmdloop(): startup
 Dec  9 12:35:34 mars master[7694]: process 7701 exited, status 75
 Dec  9 12:35:34 mars master[7694]: service syncserver pid 7701 in BUSY
 state: terminated abnormally

 here are the settings from both imapd.conf and cyrus.conf
 for both master and backup server.

 master server
 (from where everything is being copied)

 cyrus.conf

 under START section:
 syncclientcmd=/usr/local/cyrus/bin/sync_client -r -t


 imapd.conf

 #replication configurations
 sync_host: server.mydomain.com
 sync_authname: cyrus
 sync_passwd: password
 sync_machineid: 1
 sync_log: 1

 secondary
 (to where everything is bein copied)
 cyrus.conf

 SERVICES {
 syncservercmd=sync_server listen=csync





 
 Cyrus Home Page: http://cyrusimap.web.cmu.edu/
 Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
 List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html




Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html