Re: Dsync config help

[Aki Tuomi]

Hi! You can either send me the new text or I can send you the "password". 

Aki Tuomi 
Dovecot oy 


> On September 1, 2016 at 8:58 PM Jean-Luc Wasmer  wrote:
> 
> 
> Looks like I'm having a conversation with myself! lol
> 
> To avoid the timeouts, I removed the "replication_sync_timeout" setting. 
> It did work and also I don't get the "out of memory errors". I guess 
> they were related.
> I also figured the "permission denied" errors: there was a typo in one 
> of the "mode" in my config file.
> 
> 
> So to recap, here are the issues I couldn't fix:
> 
> - replication_sync_timeout generates a lot of time out error resulting 
> in out of memory errors.
> 
> - users outside the valid uid range [first_valid_uid;last_valid_uid] 
> won't be synced but will still generate a lot of errors
> 
> 
> The wiki page is of poor quality... it would need to be rewritten. I'm 
> willing to do it if someone can tell me how to go about doing that. I'n 
> not expecting any reply though...
> 
> Cheers,
> Jean-Luc
> 
> On 2016-08-31 21:01, Jean-Luc Wasmer wrote:
> > I was able to workaround the “Error: sync:
> > /var/run/dovecot/auth-userdb: Configured passdbs don’t support
> > credentials lookups (to see if user is proxied, because doveadm_port
> > is set)” problem: 
> > 
> > => I simply disabled “doveadm_port” and added the port number at the
> > end of the remote URL in “mail_replica”
> > 
> > 
> > Replication seems to work although I have regular errors (idk if they
> > are related to each other or not).
> > I’ve attached excerpts from the log files but here are the different 
> > errors:
> > 
> > 
> > - out of memory errors:
> > Aug 31 21:37:10 serverA dovecot: replicator: Panic: data stack: Out of
> > memory when allocating 268435496 bytes
> > 
> > 
> > 
> > - timeout errors:
> > 
> > Aug 31 21:36:41 serverA dovecot: dsync-server(user1): Warning:
> > replication(user1): Sync failure: Timeout in 2 secs
> > 
> > Aug 31 21:37:09 serverA dovecot: dsync-local(user1): Error: Couldn't
> > lock /home/user1/.dovecot-sync.lock: Timed out after 30 seconds
> > 
> > Aug 31 21:38:34 serverA dovecot: imap(user2): Warning:
> > replication(user2): Sync failure: Timeout in 2 secs
> > 
> > 
> > 
> > - permission denied errors:
> > 
> > Aug 31 21:37:14 serverA dovecot: dsync-server(user2): Error:
> > net_connect_unix(/var/run/dovecot/replicator-doveadm) failed:
> > Permission denied
> > 
> > 
> > 
> > 
> > 
> > I also get errors for user “nobody” with uid 65534. To avoid them I 
> > added 
> > 10-mail.conf:first_valid_uid = 2000
> > 10-mail.conf:last_valid_uid = 3000
> > 
> > 
> > 
> > but I still get some errors, different ones now:
> > 
> > 
> > Sep  1 00:13:42 shaun dovecot: doveadm(a.b.c.d,nobody): Error: Mail
> > access for users with UID 65534 not permitted (see first_valid_uid in
> > config file, uid from userdb lookup).
> > Sep  1 00:13:42 shaun dovecot: doveadm(a.b.c.d,nobody): Error:
> > dsync-server: User init failed
> > Sep  1 00:14:00 shaun dovecot: doveadm(nobody): Error: sync: Failed to
> > start remote dsync-server command: Remote exit_code=75
> > 
> > 
> > 
> > 
> > 
> > Jean-Luc
> > 
> > 
> > 
> >> On Aug 31, 2016, at 12:02 AM, Jean-Luc Wasmer  
> >> wrote:
> >> 
> >> 
> >> Hi,
> >> 
> >> 
> >> I'm trying to sync emails between server A and B.
> >> A has replicator, aggregator and doveadm.
> >> B only has doveadm for now.
> >> 
> >> 
> >> I'm getting the following error:
> >> 
> >> 
> >> Error: sync: /var/run/dovecot/auth-userdb: Configured passdbs don't 
> >> support credentials lookups (to see if user is proxied, because 
> >> doveadm_port is set)
> >> 
> >> 
> >> I'm not sure why I'm getting this error because I authenticate using 
> >> PAM 
> >> and when I run
> >> I do have a "service auth" enabled though... it's used for Postfix 
> >> SASL 
> >> to authenticate SMTP.
> >> 
> >> 
> >> I'm running
> >> 
> >> 
> >> # dovecot --version
> >> 2.2.24 (a82c823)
> >> 
> >> 
> >> 
> >> 
> >> 
> >> Cheers,
> >> Jean-Luc
> >> 
> >> 
> >> 
> >> 
> >> 
> >> PS: "replication_sync_timeout = 2s" doesn't work, I had to drop the 
> >> 's'
> >> 
> >> 
> >> 
> >> 
> >> 
> >> On 2016-08-28 17:43, Jean-Luc Wasmer wrote:
> >>> 
> >>> 
> >>> Hi,
> >>> 
> >>> 
> >>> 
> >>> 
> >>> 
> >>> I’m having a hard time with the http://wiki2.dovecot.org/Replication
> >>>  page.
> >>> 
> >>> 
> >>> 
> >>> 
> >>> 
> >>> - for a master-master setup, does the configuration need to be
> >>> mirrored on both masters?
> >>> 
> >>> 
> >>> 
> >>> 
> >>> 
> >>> 
> >>> 
> >>> 
> >>> - after aggregating unrelated sections of the wiki page, there seems
> >>> to be 3 different values for “mail_replica":
> >>> - "remote”: for SSH
> >>> - “remoteprefix": for SSH wrapper
> >>> - “tcp”: for TCP connection using the Doveadm protocol
> >>> Is this correct?
> >>> 
> >>> 
> >>> 
> >>> 
> >>> 
> >>> - what’s the purpose of the replicator VS aggregator VS doveadm 
> >>> services?
> >>> Who talks to who? Where can I find 

Re: Dsync config help

[Jean-Luc Wasmer]

Looks like I'm having a conversation with myself! lol

To avoid the timeouts, I removed the "replication_sync_timeout" setting. 
It did work and also I don't get the "out of memory errors". I guess 
they were related.
I also figured the "permission denied" errors: there was a typo in one 
of the "mode" in my config file.



So to recap, here are the issues I couldn't fix:

- replication_sync_timeout generates a lot of time out error resulting 
in out of memory errors.


- users outside the valid uid range [first_valid_uid;last_valid_uid] 
won't be synced but will still generate a lot of errors



The wiki page is of poor quality... it would need to be rewritten. I'm 
willing to do it if someone can tell me how to go about doing that. I'n 
not expecting any reply though...


Cheers,
Jean-Luc

On 2016-08-31 21:01, Jean-Luc Wasmer wrote:

I was able to workaround the “Error: sync:
/var/run/dovecot/auth-userdb: Configured passdbs don’t support
credentials lookups (to see if user is proxied, because doveadm_port
is set)” problem: 

=> I simply disabled “doveadm_port” and added the port number at the
end of the remote URL in “mail_replica”


Replication seems to work although I have regular errors (idk if they
are related to each other or not).
I’ve attached excerpts from the log files but here are the different 
errors:



- out of memory errors:
Aug 31 21:37:10 serverA dovecot: replicator: Panic: data stack: Out of
memory when allocating 268435496 bytes



- timeout errors:

Aug 31 21:36:41 serverA dovecot: dsync-server(user1): Warning:
replication(user1): Sync failure: Timeout in 2 secs

Aug 31 21:37:09 serverA dovecot: dsync-local(user1): Error: Couldn't
lock /home/user1/.dovecot-sync.lock: Timed out after 30 seconds

Aug 31 21:38:34 serverA dovecot: imap(user2): Warning:
replication(user2): Sync failure: Timeout in 2 secs



- permission denied errors:

Aug 31 21:37:14 serverA dovecot: dsync-server(user2): Error:
net_connect_unix(/var/run/dovecot/replicator-doveadm) failed:
Permission denied





I also get errors for user “nobody” with uid 65534. To avoid them I 
added 

10-mail.conf:first_valid_uid = 2000
10-mail.conf:last_valid_uid = 3000



but I still get some errors, different ones now:


Sep  1 00:13:42 shaun dovecot: doveadm(a.b.c.d,nobody): Error: Mail
access for users with UID 65534 not permitted (see first_valid_uid in
config file, uid from userdb lookup).
Sep  1 00:13:42 shaun dovecot: doveadm(a.b.c.d,nobody): Error:
dsync-server: User init failed
Sep  1 00:14:00 shaun dovecot: doveadm(nobody): Error: sync: Failed to
start remote dsync-server command: Remote exit_code=75





Jean-Luc



On Aug 31, 2016, at 12:02 AM, Jean-Luc Wasmer  
wrote:



Hi,


I'm trying to sync emails between server A and B.
A has replicator, aggregator and doveadm.
B only has doveadm for now.


I'm getting the following error:


Error: sync: /var/run/dovecot/auth-userdb: Configured passdbs don't 
support credentials lookups (to see if user is proxied, because 
doveadm_port is set)


I'm not sure why I'm getting this error because I authenticate using 
PAM 

and when I run
I do have a "service auth" enabled though... it's used for Postfix 
SASL 

to authenticate SMTP.


I'm running


# dovecot --version
2.2.24 (a82c823)





Cheers,
Jean-Luc





PS: "replication_sync_timeout = 2s" doesn't work, I had to drop the 
's'






On 2016-08-28 17:43, Jean-Luc Wasmer wrote:



Hi,





I’m having a hard time with the http://wiki2.dovecot.org/Replication
 page.





- for a master-master setup, does the configuration need to be
mirrored on both masters?








- after aggregating unrelated sections of the wiki page, there seems
to be 3 different values for “mail_replica":
- "remote”: for SSH
- “remoteprefix": for SSH wrapper
- “tcp”: for TCP connection using the Doveadm protocol
Is this correct?





- what’s the purpose of the replicator VS aggregator VS doveadm 
services?
Who talks to who? Where can I find documentation about their
configuration (i.e. not examples)





- most examples are for a single vmail user setup… what should be 
done

when using system accounts?





Thanks,
Jean-Luc

Re: Dsync config help

[Jean-Luc Wasmer]

I was able to workaround the “Error: sync: /var/run/dovecot/auth-userdb: 
Configured passdbs don’t support credentials lookups (to see if user is 
proxied, because doveadm_port is set)” problem: 

=> I simply disabled “doveadm_port” and added the port number at the end of the 
remote URL in “mail_replica”


Replication seems to work although I have regular errors (idk if they are 
related to each other or not).
I’ve attached excerpts from the log files but here are the different errors:


- out of memory errors:
Aug 31 21:37:10 serverA dovecot: replicator: Panic: data stack: Out of memory 
when allocating 268435496 bytes



- timeout errors:

Aug 31 21:36:41 serverA dovecot: dsync-server(user1): Warning: 
replication(user1): Sync failure: Timeout in 2 secs

Aug 31 21:37:09 serverA dovecot: dsync-local(user1): Error: Couldn't lock 
/home/user1/.dovecot-sync.lock: Timed out after 30 seconds

Aug 31 21:38:34 serverA dovecot: imap(user2): Warning: replication(user2): Sync 
failure: Timeout in 2 secs



- permission denied errors:

Aug 31 21:37:14 serverA dovecot: dsync-server(user2): Error: 
net_connect_unix(/var/run/dovecot/replicator-doveadm) failed: Permission denied





I also get errors for user “nobody” with uid 65534. To avoid them I added 
10-mail.conf:first_valid_uid = 2000
10-mail.conf:last_valid_uid = 3000



but I still get some errors, different ones now:


Sep  1 00:13:42 shaun dovecot: doveadm(a.b.c.d,nobody): Error: Mail access for 
users with UID 65534 not permitted (see first_valid_uid in config file, uid 
from userdb lookup).
Sep  1 00:13:42 shaun dovecot: doveadm(a.b.c.d,nobody): Error: dsync-server: 
User init failed
Sep  1 00:14:00 shaun dovecot: doveadm(nobody): Error: sync: Failed to start 
remote dsync-server command: Remote exit_code=75





Jean-Luc



> On Aug 31, 2016, at 12:02 AM, Jean-Luc Wasmer  wrote:
> 
> 
> Hi,
> 
> 
> I'm trying to sync emails between server A and B.
> A has replicator, aggregator and doveadm.
> B only has doveadm for now.
> 
> 
> I'm getting the following error:
> 
> 
> Error: sync: /var/run/dovecot/auth-userdb: Configured passdbs don't 
> support credentials lookups (to see if user is proxied, because 
> doveadm_port is set)
> 
> 
> I'm not sure why I'm getting this error because I authenticate using PAM 
> and when I run
> I do have a "service auth" enabled though... it's used for Postfix SASL 
> to authenticate SMTP.
> 
> 
> I'm running
> 
> 
> # dovecot --version
> 2.2.24 (a82c823)
> 
> 
> 
> 
> 
> Cheers,
> Jean-Luc
> 
> 
> 
> 
> 
> PS: "replication_sync_timeout = 2s" doesn't work, I had to drop the 's'
> 
> 
> 
> 
> 
> On 2016-08-28 17:43, Jean-Luc Wasmer wrote:
>> 
>> 
>> Hi,
>> 
>> 
>> 
>> 
>> 
>> I’m having a hard time with the http://wiki2.dovecot.org/Replication
>>  page.
>> 
>> 
>> 
>> 
>> 
>> - for a master-master setup, does the configuration need to be
>> mirrored on both masters?
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> - after aggregating unrelated sections of the wiki page, there seems
>> to be 3 different values for “mail_replica":
>> - "remote”: for SSH
>> - “remoteprefix": for SSH wrapper
>> - “tcp”: for TCP connection using the Doveadm protocol
>> Is this correct?
>> 
>> 
>> 
>> 
>> 
>> - what’s the purpose of the replicator VS aggregator VS doveadm 
>> services?
>> Who talks to who? Where can I find documentation about their
>> configuration (i.e. not examples)
>> 
>> 
>> 
>> 
>> 
>> - most examples are for a single vmail user setup… what should be done
>> when using system accounts?
>> 
>> 
>> 
>> 
>> 
>> Thanks,
>> Jean-Luc
>> 
Aug 31 21:36:41 serverA dovecot: dsync-server(user1): Warning: 
replication(user1): Sync failure: Timeout in 2 secs
Aug 31 21:37:09 serverA dovecot: dsync-local(user1): Error: Couldn't lock 
/home/user1/.dovecot-sync.lock: Timed out after 30 seconds
Aug 31 21:37:10 serverA dovecot: replicator: Panic: data stack: Out of memory 
when allocating 268435496 bytes
Aug 31 21:37:10 serverA dovecot: replicator: Fatal: master: 
service(replicator): child 76676 killed with signal 6 (core not dumped - set 
service replicator { drop_priv_before_exec=yes })
Aug 31 21:37:14 serverA dovecot: dsync-server(user2): Error: 
net_connect_unix(/var/run/dovecot/replicator-doveadm) failed: Permission denied
Aug 31 21:37:18 serverA dovecot: dsync-server(user1): Error: 
net_connect_unix(/var/run/dovecot/replicator-doveadm) failed: Permission denied
Aug 31 21:38:34 serverA dovecot: imap(user2): Warning: replication(user2): Sync 
failure: Timeout in 2 secs
Aug 31 21:38:46 serverA dovecot: replicator: Panic: data stack: Out of memory 
when allocating 268435496 bytes
Aug 31 21:38:46 serverA dovecot: replicator: Fatal: master: 
service(replicator): child 76734 killed with signal 6 (core not dumped - set 
service replicator { drop_priv_before_exec=yes })
Aug 31 21:38:58 serverA dovecot: dsync-server(user2): Error: 
net_connect_unix(/var/run/dovecot/replicator-doveadm) failed: Permission denied
Aug 31 21:39:21 serverA 

Re: Dsync config help

[Jean-Luc Wasmer]

Hi,

I'm trying to sync emails between server A and B.
A has replicator, aggregator and doveadm.
B only has doveadm for now.

I'm getting the following error:

Error: sync: /var/run/dovecot/auth-userdb: Configured passdbs don't 
support credentials lookups (to see if user is proxied, because 
doveadm_port is set)


I'm not sure why I'm getting this error because I authenticate using PAM 
and when I run
I do have a "service auth" enabled though... it's used for Postfix SASL 
to authenticate SMTP.


I'm running

# dovecot --version
2.2.24 (a82c823)


Cheers,
Jean-Luc


PS: "replication_sync_timeout = 2s" doesn't work, I had to drop the 's'


On 2016-08-28 17:43, Jean-Luc Wasmer wrote:

Hi,


I’m having a hard time with the http://wiki2.dovecot.org/Replication
 page.


- for a master-master setup, does the configuration need to be
mirrored on both masters?



- after aggregating unrelated sections of the wiki page, there seems
to be 3 different values for “mail_replica":
- "remote”: for SSH
- “remoteprefix": for SSH wrapper
- “tcp”: for TCP connection using the Doveadm protocol
Is this correct?


- what’s the purpose of the replicator VS aggregator VS doveadm 
services?

Who talks to who? Where can I find documentation about their
configuration (i.e. not examples)


- most examples are for a single vmail user setup… what should be done
when using system accounts?


Thanks,
Jean-Luc

Dsync config help

[Jean-Luc Wasmer]

Hi,


I’m having a hard time with the http://wiki2.dovecot.org/Replication
 page.


- for a master-master setup, does the configuration need to be mirrored on both 
masters?



- after aggregating unrelated sections of the wiki page, there seems to be 3 
different values for “mail_replica":
- "remote”: for SSH
- “remoteprefix": for SSH wrapper
- “tcp”: for TCP connection using the Doveadm protocol
Is this correct?


- what’s the purpose of the replicator VS aggregator VS doveadm services?
Who talks to who? Where can I find documentation about their configuration 
(i.e. not examples)


- most examples are for a single vmail user setup… what should be done when 
using system accounts?


Thanks,
Jean-Luc