Re: Disable Dovecot LDA

[Remko Lodder]

If both postfix’ in your example are the same server, then that’s not possible 
indeed.
Consider it has one brain, so it has to decide: Do I know this domain or do I 
need to
send it upstream (either to the MX of the target domain, or to a configured 
relay).

In case it’s a domain postfix hosts, a delivery attempt will be made directly 
via LMTP/LDA.
It does not go out (since it’s known local). 

In case the domain is not hosted locally, it will be forwarded to a relay, and 
that has to decide
where to send it next. If your machine is the best known option for the target 
domain, then it
will be send there. But if that is the same postfix server, that’s an issue: 
because the domain is
not known locally ,and thus it should be rejected. Message = lost.

In case you have 2 postfix instances, where A is on the top side of your image, 
then it can be configured
to route all email to relay, and that has a configuration that send’s it to 
instance B (the bottom); that can have
configuration that treats the target domain as local, and thus starts an 
delivery attempt to LMTP/LDA.

Does that help a bit?

Cheers
Remko

> On 2 Apr 2020, at 21:29, Adam Raszkiewicz  wrote:
> 
> So there is no way to skip Postfix for incoming messages and go directly to 
> LMTP/LDA but I need two instances of Postfix in that case?
> 
> Regarding your second question - it is required to be complaint with Direct 
> Trust. 
> 
> Thanks,
> Adam
> 
> On 4/2/20, 3:07 PM, "dovecot on behalf of Juri Haberland" 
>  wrote:
> 
>On 02/04/2020 15:18, Adam Raszkiewicz wrote:
>> Desired flow looks like:
>> 
>> Dovecot -> Postfix --> Relay Server -┐
>> Dovecot <-- LMTP/LDA <-- Postfix <-┘
> 
>This mail flow cannot work with one Postfix instance. Either Postfix
>knows that "localdomain.com" is local and should be delivered to the
>LDA, in which case it won't be forwarded to the relay server, or Postfix
>does not know that "localdomain.com" is a local address and therefor
>forwards it the relay server, but than it will do that anytime it sees
>"localdomain.com".
> 
>Only possibility is to run two instances of Postfix.
> 
>The real question is:
>Why do you want this mail flow? Where is the benefit in sending a local
>mail out to a relay server only to get it back and deliver it?
> 
> 
>Cheers,
>  Juri
> 
> 

Re: encryption issue in outlook 2016

[Remko Lodder]

dovecot-support is not windows support. You ought to know how to handle the 
windows part yourself.
If the issue is within dovecot, then I am sure Aki and his coworkers are 
willing to help you the best
they can.

Cheers
Remko

> On 14 Sep 2018, at 12:47, ChandranManikandan  wrote:
> 
> Hi Aki,
> 
> I don't purchase anything.
> It's installed in the server when install qmailtoaster.
> Please suggest the procedure.
> 
> On Fri, Sep 14, 2018 at 5:15 PM Aki Tuomi  > wrote:
> The certificate has wrong subject, the certificate has a local issuer which 
> isn't probably installed to your machine's trust store. So you should 
> probably make sure the cert has correct subject (server's DNS name) and that 
> you install the issuer as trusted issuer, or get a cert from a trusted 
> provider.
> Aki
> 
> On 14.09.2018 11:27, ChandranManikandan wrote:
>> Hi Friends,
>> 
>> Am facing the serious issue in outlook 2016.
>> 
>> When i tried to configure the email in outlook 2016 it's asking encrypted 
>> connection and checked automatically and if not available then it showing 
>> unencrypted. but finally unable to configure the email account in outlook 
>> 2016 and not permitted.
>> 
>> Do i need to update any in the server.
>> 
>> Am using COS 6 32 and 64 bit servers with qmailtoaster + dovecot 2.2.7.
>> 
>> Currently configured in outlook 2010 it's working without any issue.
>> And always showing in outlook 2010 when i open the account.
>> Do you want to continue yes or no for the certification.
>> Because i have configured in advanced selected Auto in outlook 2010.
>> If i select none it was not working the email.
>> 
>> Anyone facing the same issue.
>> 
>> How do i rectify it in outlook 2016 and how can i stop the certificate 
>> warning message in outlook.
>> 
>> If possible to use without encryption and what's the configuration.
>> 
>> Please advice and let me know the process.
>> 
>> Am seriously cannot access the email from outlook.
>> I have attached the message while open outlook. If i click yes then it will 
>> open
>> 
>> Is the certificate expired. How do renew it or reload it with the fresh 
>> certificate.
>> 
>> 
>> --
>> Thanks & Best Regards,
>> Manikandan.C
> 
> 
> 
> --
> Thanks & Best Regards,
> Manikandan.C



signature.asc
Description: Message signed with OpenPGP

Re: Sharing Mailbox between users using IMAP

[Remko Lodder]

> On 29 Jun 2018, at 09:42, Luca Bertoncello  wrote:
> 
> Hi list!
> 
> I have an account (let's say i...@mydomain.com) that should be read from more 
> people.
> These people does NOT have an account on the server.
> Currently i...@mydomain.com is a forward to their addresses, but of course 
> this solution has a huge problem: if info@ receives spam that the server does 
> not recognize, the server forwards spam...
> 
> Now I want to solve this problem and I had the idea to change info@ from 
> "forward" to "local account".
> Then, I can read the E-Mails via IMAP.
> 
> Now the question: if I have 5-6 people accessing the account via IMAP, has 
> Dovecot (2.2.13 from Debian repositories) problems?
> 
> Is there other solution, if the users don't want to have an account on the 
> server?
> 
> Thanks a lot for your suggestion
> 
> Luca Bertoncello
> (lucab...@lucabert.de)
> 

Emails can only be read if they are authenticated / authorized in someway to 
access the store. That means you might need to share the info@ credentials with 
the other
people so that they can read it over imap or webmail etc.

Cheers,
remko


signature.asc
Description: Message signed with OpenPGP

Re: 2.3.1 Replication is throwing scary errors

[Remko Lodder]

> On 7 Jun 2018, at 07:21, Reuben Farrelly  wrote:
> 
> Still not quite right for me.
> 
> Jun  7 15:11:33 thunderstorm.reub.net dovecot: doveadm: Error: 
> dsync(lightning.reub.net): I/O has stalled, no activity for 600 seconds (last 
> sent=mail, last recv=mail (EOL))
> Jun  7 15:11:33 thunderstorm.reub.net dovecot: doveadm: Error: Timeout during 
> state=sync_mails (send=mails recv=recv_last_common)
> 
> I'm not sure if there is an underlying replication error or if the message is 
> just cosmetic, though.
> 
> Reuben

Hi,

Admittedly I have had a few occurences of this behaviour as well last night. It 
happens more sporadic now and seems to be a conflict with my user settings. (My 
users
get added twice to the system, user-domain.tld and u...@domain.tld 
<mailto:u...@domain.tld>, both are being replicated, the noreplicate flag is 
not yet honored in the version I am using so I cannot
bypass that yet).

I do see messages that came on the other machine on the machine that I am using 
to read these emails. So replication seems to work in that regard (where it 
obviously
did not do that well before).

Cheers
Remko

> 
> 
> On 7/06/2018 4:55 AM, Remko Lodder wrote:
>> Hi Timo,
>> Yes this seems to work fine so far. I’ll ask the people to add it to the 
>> current FreeBSD version..
>> Cheers
>> Remko
>>> On 6 Jun 2018, at 19:34, Timo Sirainen mailto:t...@iki.fi>> 
>>> wrote:
>>> 
>>> Should be fixed by 
>>> https://github.com/dovecot/core/commit/a952e178943a5944255cb7c053d970f8e6d49336
>>> 



signature.asc
Description: Message signed with OpenPGP

Re: 2.3.1 Replication is throwing scary errors

[Remko Lodder]

Hi Timo,

Yes this seems to work fine so far. I’ll ask the people to add it to the 
current FreeBSD version..

Cheers
Remko

> On 6 Jun 2018, at 19:34, Timo Sirainen  wrote:
> 
> Should be fixed by 
> https://github.com/dovecot/core/commit/a952e178943a5944255cb7c053d970f8e6d49336
>  
> 
> 



signature.asc
Description: Message signed with OpenPGP

Re: 2.3.1 Replication is throwing scary errors

[Remko Lodder]

> On 31 May 2018, at 17:52, Michael Grimm  wrote:
> 
> Reuben Farrelly  wrote:
> 
>> Checking in - this is still an issue with 2.3-master as of today (2.3.devel 
>> (3a6537d59)).
> 
> That doesn't sound good, because I did hope that someone has been working on 
> this issue ...
> 
>> I haven't been able to narrow the problem down to a specific commit. The 
>> best I have been able to get to is that this commit is relatively good (not 
>> perfect but good enough):
>> 
>> d9a1a7cbec19f4c6a47add47688351f8c3a0e372 (from Feb 19, 2018)
>> 
>> whereas this commit:
>> 
>> 6418419ec282c887b67469dbe3f541fc4873f7f0 (From Mar 12, 2018)
>> 
>> is pretty bad.  Somewhere in between some commit has caused the problem 
>> (which may have been introduced earlier) to get much worse.
> 
> Thanks for the info.
> 
>> There seem to be a handful of us with broken systems who are prepared to 
>> assist in debugging this and put in our own time to patch, test and get to 
>> the bottom of it, but it is starting to look like we're basically on our own.
> 
> I wonder if there is anyone running a 2.3 master-master replication scheme 
> *without* running into this issue? Please let us know: yes, 2.3 master-master 
> replication does run as rock-stable as in 2.2.
> 
> Anyone?
> 
> I would love to get some feedback from the developers regarding:
> 
> #) are commercial customers of yours running 2.3 master-master replication 
> without those issues reported in this thread?
> #) do you get reports about these issues outside this ML as well?
> #) and ...
> 
>> What sort of debugging, short of bisecting 100+ patches between the commits 
>> above, can we do to progress this?
> 
> … what kind of debugging do you suggest?

Aki sent me over some patches recently and I have build a custom package for it 
for FreeBSD. It’s in my pkg repo which I can forward you if you want it.
You need to add some lines to the logging thing and then trace those and 
collaborate with the dovecot community/developers. I did not have yet found
the time to actively persue this due to other things on my head. Sorry for 
that. I hope to do this “soon” but I dont want to pin myself to a commitment 
that
I might not be able to make :)

Cheers
Remko

> 
> Regards,
> Michael


signature.asc
Description: Message signed with OpenPGP

Re: 2.3.1 Replication is throwing scary errors

[Remko Lodder]

> On 4 Apr 2018, at 01:34, Reuben Farrelly  wrote:
> 
> Hi,
> 
>> --
>> Message: 2
>> Date: Mon, 2 Apr 2018 22:06:07 +0200
>> From: Michael Grimm 
>> To: Dovecot Mailing List 
>> Subject: 2.3.1 Replication is throwing scary errors
>> Message-ID: <29998016-d62f-4348-93d1-613b13da9...@ellael.org>
>> Content-Type: text/plain;charset=utf-8
>> Hi
>> [This is Dovecot 2.3.1 at FreeBSD STABLE-11.1 running in two jails at 
>> distinct servers.]
>> I did upgrade from 2.2.35 to 2.3.1 today, and I do become pounded by error 
>> messages at server1 (and vice versa at server2) as follows:
>>  | Apr  2 17:12:18  server1.lan dovecot: doveadm: Error: 
>> dsync(server2.lan): I/O has stalled, \
>>  no activity for 600 seconds (last sent=mail_change, last 
>> recv=mail_change (EOL))
>>  | Apr  2 17:12:18  server1.lan dovecot: doveadm: Error: 
>> Timeout during state=sync_mails \
>>  (send=changes recv=mail_requests)
>>  [?]
>>  | Apr  2 18:59:03  server1.lan dovecot: doveadm: Error: 
>> dsync(server2.lan): I/O has stalled, \
>>  no activity for 600 seconds (last sent=mail, last recv=mail 
>> (EOL))
>>  | Apr  2 18:59:03  server1.lan dovecot: doveadm: Error: 
>> Timeout during state=sync_mails \
>>  (send=mails recv=recv_last_common)
>> I cannot see in my personal account any missing replications, *but* I 
>> haven't tested this thoroughly enough. I do have customers being serviced at 
>> these productive servers, *thus* I'm back to 2.2.35 until I do understand or 
>> have learned what is going on.
>> Any ideas/feedback?
>> FYI: I haven't seen such errors before. Replication has been working for 
>> years now, without any glitches at all.
>> Regards,
>> Michael
> 
> It's not just you.  This issue hit me recently, and it was impacting 
> replication noticeably.  I am following git master-2.3 .
> 
> 

I am seeing the same as Michael Grimm also on FreeBSD-11.
You’ll also notice in doveadm replicator status ‘*’ that the failed flag is 
raised for those users and that
there are processes just hanging forever when those logs start to appear:

   45949   0.0  0.047888  13276  -  I20:200:00.10 
doveadm-server: [ Verwijderde items send:mail_requests recv:changes] 
(doveadm-server)
 45964   0.0  0.049860  11608  -  I20:200:00.05 
doveadm-server: [IP6  INBOX import:1/3] (doveadm-server)
 45965   0.0  0.158256  19820  -  I20:200:00.11 
doveadm-server: [IP6  INBOX import:16/18] (doveadm-server)
 46480   0.0  0.053536  16288  -  I20:220:00.08 
doveadm-server: [IP6  INBOX import:4/6] (doveadm-server)
 46745   0.0  0.051496  14184  -  I20:220:00.07 
doveadm-server: [IP6  INBOX import:5/6] (doveadm-server)

I also reverted to 2.2.35 because I started to get complaints from my users 
that mail was missing.

Cheers
Remko



signature.asc
Description: Message signed with OpenPGP

Re: Cannot delete IMAP Mail-Folder in Trash

[Remko Lodder]

> On 5 Feb 2018, at 17:32, Aki Tuomi  > wrote:
> 
> 
>> On February 5, 2018 at 6:16 PM Gabriel Kaufmann > > wrote:
>> 
>> 
>> Doesn't anyone have an idea?
>> 
>> It looks like Dovecot and/or Thunderbird simply "ignore" sub-folders of
>> IMAP-Folder in Trash/ as they are also not shown in Trash-Folder.
>> Sub-Folder for INBOX and other IMAP-Folders work perfect (using
>> layout=fs Mailbox).
>> 
>> Why doesn't this work for Trash?
>> 
>> Best regards
>> 
>> Gabriel Kaufmann
>> 
> 
> Can you try running this as root and provide output?
> 
> doveadm -Dv -o mail_debug=yes mailbox delete -u username Trash/Somefolder
> 
> Aki


I had the same with Mac Mail.app. I decided to rm -rf the sdbox file on the 
master and replica and I won.
It was persistant though and I think it had to do with the following:

I let my mail deliver in yearboxes, which have many many subfolders, some are 
just “placeholders”. They appear different in the view.
Regular mailboxes are “dark” in colors, placeholder boxes are “white” in colors 
(see attachment), where winkels is the “placeholder” folder and “action” is the 
regular mailbox in which mail gets delivered.

I could not get rid of those “winkels” kind of folders”. Only by force removing 
them from the filesystem hierarchie.

Hope this helps a bit :)

Cheers
Remko





signature.asc
Description: Message signed with OpenPGP

user selection for replication

[Remko Lodder]

Hi,

I am running a setup where I have a multitude of users.
Usernames are formed in both user-domain.tld and u...@domain.tld.
One to be able to login with the @ account to dovecot / webmail etc
and the - account to be able to deliver email to it.

Virtualmin does it like that for example.

This is fine for logging in to accounts and telling postfix via lmtp which
users are valid. But I do not want the - AND @ account to both synchronise.

How can I select which users can replicate and then ideally like I can do
in ldap ‘uid=*@*’ for example?

Thanks
Remko




signature.asc
Description: Message signed with OpenPGP

Replication oddities - different sizes between replicated nodes

[Remko Lodder]

Dear Dovecot community,

As discussing on IRC with Aki, I have the following thing happening:

I got an alert from my mailservice last night that I was running over quota. My 
quota reached almost 91% (coming from 30%) in one day.
I do not recall receiving so much email (we talk about multiple gb’s additional 
space usage), so something was acting up.

When I logged into my mailservers (I’ll describe my setup later on), it 
mentioned that my FreeBSD mailbox was reaching 12gb of space usage.
While the seeming replicated machine (named A) only had 100mb of space in use 
(which, with the amount of messages in the various mailboxes
is consistent with what I would expect).

I found out that the “Junk” mailbox was the ‘major' issue. I vaporized the 
replicated (12GB) mailbox and issued a ‘doveadm sync -u $user -N -U tcps:hostB`
so that I would expect the hostB to reach around 100mb-ish of email. I let it 
go for a little and saw that the mail was done. When I issued a simple ‘df -h’ 
on
the directory, it was already multiple 100mb’s large and growing. It grew back 
to multiple GB in little time. Again the Junk mailbox was the culprit. In the 
mean
time I noticed my mailclient downloading 44k mails in that mailbox (where there 
were only a few in the Junk mailbox normally).

I again vaporized the entire mailbox on hostB and cleaned out Junk and issued a 
manual synchronisation. It again grew quickly.

Aki did an investigation with me and noticed that my rspamc script was not very 
well written. I rewrote it as suggested (As written on the site).

This morning I continued with the investigation and it was again large on hostB 
where it was still 100mb-ish on hostA. I vaporized host B again and
issued the manual sync with -Dv included per request of Aki. After letting it 
run for a little hostA grew to 300mb and hostB restarted to reach 1.2GB and
remained static there. After half an hour or so the mailbox on hostA grew to 
900mb and stayed there.

I investigated the mailboxes and saw 8 times the same email. Cleaning them up 
with doveadm deduplicate -u $user mailbox ‘*’ made sure a lot of email
was cut from both hosts. I am now in the state where:

hostA: 268MB
hostB: 579MB

Both are for the same mailbox (I suspect that other mailboxes have similiar 
issues).
When I look at the amount of messages counted by Apple Mail in the particular 
mailbox I see 1399 emails. The amount of ‘u’ files is 14722 on hostA and 23590 
on hostB.

I do not understand the difference from my mail client and the amount of u 
files.

from mutt: Mutt: =mailbox [Msgs:1415 New:1 Flag:1  24M]-, which is roughly the 
same as Apple mail reports.

I am a bit lost on where to look next, I would suspect that the synchronised 
mailboxes would be similar in size, but not the one 2 as large, or before that 
it grew and grew
and grew.

My personal feeling is that the sync takes place, but somehow it is not 
registering that a message had been synchronised and does that again, and 
again, and again till
the message gets synchronised right and the acknowledge is received and it 
stays the same. That would at least support the 8times the same email that I 
saw at first
and might also support the roughly double as big size of the other host (one 
sync goes wrong, the next sync went right and got acked). At the bottom I also 
referenced
two previous emails of mine which report similar oddities. Messages that were 
removed on hostA got back within seconds as “new message” . My feeling there is 
that
the hostB has the ‘failed sync copy with a different uid/guid’ and tells hostA 
that there is a new message and sends it over.

My doveconf -n for hostA (hostB follows), I did sanitize a bit wrt. IP 
addresses and hostnames:

 2.2.33.2 (d6601f4ec): /usr/local/etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.21 (92477967)
# OS: FreeBSD 11.1-RELEASE amd64
auth_mechanisms = plain login
disable_plaintext_auth = no
doveadm_password =  # hidden, use -P to show it
lda_mailbox_autocreate = yes
lda_mailbox_autosubscribe = yes
lmtp_save_to_detail_mailbox = yes
login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c %k
mail_debug = yes
mail_fsync = always
mail_location = sdbox:~/sdbox
mail_plugins = " quota notify replication"
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character 
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy 
include variables body enotify environment mailbox date index ihave duplicate 
mime foreverypart extracttext imapsieve vnd.dovecot.imapsieve
namespace {
  inbox = yes
  location =
  mailbox Drafts {
auto = subscribe
special_use = \Drafts
  }
  mailbox Junk {
special_use = \Junk
  }
  mailbox Sent {
auto = subscribe
special_use = \Sent
  }
  mailbox "Sent Messages" {
special_use = \Sent
  }
  mailbox Spam {
auto = subscribe
special_use = \Junk
  }
  mailbox Trash {
auto = subscribe
special_use = \Trash
  

Re: Question regarding replication - duplicate emails

[Remko Lodder]

Hi Aki,

Uhm well, that’s pretty hard not to do so…
Mail.app only has one “Delete” option and that triggers the situation.
My support application fetches the email and then deletes it (basically the 
same as Mail.app).
My customers use a variety of MUA’s and they see the same behaviour..

I think the “notification” for the removal is not yet on the other machine, but 
it sees something
changing and thinks “well I have this new message, that you do not have, sync 
it”.

Thanks,
Remko

> On 21 Oct 2017, at 15:06, Aki Tuomi  wrote:
> 
> Maybe you could not delete the message instantly, but keep track of last seen 
> UID. UIDs increase monotonically, so you can rest assured that next arriving 
> email has larger UID.
> 
> Then you can bulk delete mails that are older than one day.
> 
> Aki
> 
>> On October 20, 2017 at 8:01 PM Remko Lodder  wrote:
>> 
>> 
>> Hi,
>> 
>> Anyone has suggestions? The situation also happends when I “delete” a 
>> message from my Phone.
>> It seems to get replicated instantly and the just deleted email is back in 
>> the mailbox again.
>> 
>> I did remove HA Proxy support in the meantime to rule that out and I have 
>> enabled the default
>> replication_max_conns.
>> 
>> thanks
>> Remko
>> 
>>> On 13 Oct 2017, at 11:56, Remko Lodder  wrote:
>>> 
>>> Dear Dovecot and community,
>>> 
>>> We run a small email service for our customers, based on two machines that 
>>> are made “redundant or clustered” by using the replication feature of 
>>> Dovecot.
>>> This works well, for most emails.
>>> 
>>> Sometimes the following happends:
>>> 
>>> Email to our support database arrives at the inbox.
>>> Every period a cronjob looks into that mailbox and parses the information 
>>> and makes a support ticket from that message.
>>> The cronjob deletes the email afterwards and sees whether there are more 
>>> new mails or not.
>>> 
>>> In between the deletion and the (next) check, the email gets “resynced” 
>>> again from the “other machine”, and is seen as a new
>>> email by the cronjob and gets handled again. So basically we get two 
>>> tickets for one email. After that second run the email is
>>> no longer replicated.
>>> 
>>> This behaviour can also be seen via webmail and email clients, this 
>>> suggests that the replication might not be aware (enough) that
>>> the replication is occurring or that an item already had been synced from A 
>>> to B (and does not need to get back from B to A after A
>>> deleted it).
>>> 
>>> Is there a way to fiddle with the acknowledgement timing or give the 
>>> processes some more space/time to get on par with eachother?
>>> 
>>> Below is the configuration of machine B, they are synchronised through 
>>> puppet, so only the hostname and IP addresses are different.
>>> (so for replication, A has: tcps:hostname_of_b:12346 and B has 
>>> tcps:hostname_of_a:12346). Also note that HAproxy is prepared but
>>> not in use at all.
>>> 
>>> # 2.2.33.1 (e9afa7f18): /usr/local/etc/dovecot/dovecot.conf
>>> # Pigeonhole version 0.4.20 (7cd71ba)
>>> # OS: FreeBSD 11.1-RELEASE amd64
>>> auth_mechanisms = plain login
>>> disable_plaintext_auth = no
>>> doveadm_password =  # hidden, use -P to show it
>>> haproxy_trusted_networks = IPv4_Haproxy IPv6_Haproxy
>>> lda_mailbox_autocreate = yes
>>> lda_mailbox_autosubscribe = yes
>>> lmtp_save_to_detail_mailbox = yes
>>> mail_fsync = always
>>> mail_location = sdbox:~/sdbox
>>> mail_plugins = " quota notify replication"
>>> managesieve_notify_capability = mailto
>>> managesieve_sieve_capability = fileinto reject envelope encoded-character 
>>> vacation subaddress comparator-i;ascii-numeric relational regex imap4flags 
>>> copy include variables body enotify environment mailbox date index ihave 
>>> duplicate mime foreverypart extracttext imapsieve vnd.dovecot.imapsieve
>>> namespace {
>>> inbox = yes
>>> location =
>>> mailbox Drafts {
>>>   auto = subscribe
>>>   special_use = \Drafts
>>> }
>>> mailbox Junk {
>>>   special_use = \Junk
>>> }
>>> mailbox Sent {
>>>   auto = subscribe
>>>   special_use = \Sent
>>> }
>>> mailbox "Sent Messages" {
>>>   special_use = \Sent
>>> }
>>> mailbox Spam {
>>>   auto = subscribe

Re: Question regarding replication - duplicate emails

[Remko Lodder]

Hi,

Anyone has suggestions? The situation also happends when I “delete” a message 
from my Phone.
It seems to get replicated instantly and the just deleted email is back in the 
mailbox again.

I did remove HA Proxy support in the meantime to rule that out and I have 
enabled the default
replication_max_conns.

thanks
Remko

> On 13 Oct 2017, at 11:56, Remko Lodder  wrote:
> 
> Dear Dovecot and community,
> 
> We run a small email service for our customers, based on two machines that 
> are made “redundant or clustered” by using the replication feature of Dovecot.
> This works well, for most emails.
> 
> Sometimes the following happends:
> 
> Email to our support database arrives at the inbox.
> Every period a cronjob looks into that mailbox and parses the information and 
> makes a support ticket from that message.
> The cronjob deletes the email afterwards and sees whether there are more new 
> mails or not.
> 
> In between the deletion and the (next) check, the email gets “resynced” again 
> from the “other machine”, and is seen as a new
> email by the cronjob and gets handled again. So basically we get two tickets 
> for one email. After that second run the email is
> no longer replicated.
> 
> This behaviour can also be seen via webmail and email clients, this suggests 
> that the replication might not be aware (enough) that
> the replication is occurring or that an item already had been synced from A 
> to B (and does not need to get back from B to A after A
> deleted it).
> 
> Is there a way to fiddle with the acknowledgement timing or give the 
> processes some more space/time to get on par with eachother?
> 
> Below is the configuration of machine B, they are synchronised through 
> puppet, so only the hostname and IP addresses are different.
> (so for replication, A has: tcps:hostname_of_b:12346 and B has 
> tcps:hostname_of_a:12346). Also note that HAproxy is prepared but
> not in use at all.
> 
> # 2.2.33.1 (e9afa7f18): /usr/local/etc/dovecot/dovecot.conf
> # Pigeonhole version 0.4.20 (7cd71ba)
> # OS: FreeBSD 11.1-RELEASE amd64
> auth_mechanisms = plain login
> disable_plaintext_auth = no
> doveadm_password =  # hidden, use -P to show it
> haproxy_trusted_networks = IPv4_Haproxy IPv6_Haproxy
> lda_mailbox_autocreate = yes
> lda_mailbox_autosubscribe = yes
> lmtp_save_to_detail_mailbox = yes
> mail_fsync = always
> mail_location = sdbox:~/sdbox
> mail_plugins = " quota notify replication"
> managesieve_notify_capability = mailto
> managesieve_sieve_capability = fileinto reject envelope encoded-character 
> vacation subaddress comparator-i;ascii-numeric relational regex imap4flags 
> copy include variables body enotify environment mailbox date index ihave 
> duplicate mime foreverypart extracttext imapsieve vnd.dovecot.imapsieve
> namespace {
>  inbox = yes
>  location =
>  mailbox Drafts {
>auto = subscribe
>special_use = \Drafts
>  }
>  mailbox Junk {
>special_use = \Junk
>  }
>  mailbox Sent {
>auto = subscribe
>special_use = \Sent
>  }
>  mailbox "Sent Messages" {
>special_use = \Sent
>  }
>  mailbox Spam {
>auto = subscribe
>special_use = \Junk
>  }
>  mailbox Trash {
>auto = subscribe
>special_use = \Trash
>  }
>  prefix =
>  separator = .
> }
> passdb {
>  driver = pam
> }
> plugin {
>  imapsieve_mailbox1_before = 
> file:/usr/local/lib/dovecot/sieve/report-spam.sieve
>  imapsieve_mailbox1_causes = COPY
>  imapsieve_mailbox1_name = Spam
>  imapsieve_mailbox2_before = 
> file:/usr/local/lib/dovecot/sieve/report-ham.sieve
>  imapsieve_mailbox2_causes = COPY
>  imapsieve_mailbox2_from = Spam
>  imapsieve_mailbox2_name = *
>  mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename
>  mail_log_fields = uid box msgid size
>  mail_replica = tcps:hostname_of_machine_a:12346
>  sieve = ~/.dovecot.sieve
>  sieve_after = /usr/local/etc/dovecot/sieve/after/
>  sieve_before = /usr/local/etc/dovecot/sieve/global/
>  sieve_pipe_bin_dir = /usr/local/lib/dovecot/sieve
>  sieve_plugins = sieve_imapsieve sieve_extprograms
>  sieve_vacation_dont_check_recipient = yes
> }
> postmaster_address = postmas...@ourdomain.tld
> protocols = imap pop3 lmtp sieve
> replication_max_conns = 100
> service aggregator {
>  fifo_listener replication-notify-fifo {
>mode = 0666
>  }
>  unix_listener replication-notify {
>mode = 0666
>  }
> }
> service auth {
>  unix_listener /var/spool/postfix/private/auth {
>mode = 0666
>  }
> }
> service doveadm {
>  inet_listener {
>port = 12346
>ssl = yes
>  }
> }
> service imap-login {
>  inet_listene

Question regarding replication - duplicate emails

[Remko Lodder]

Dear Dovecot and community,

We run a small email service for our customers, based on two machines that are 
made “redundant or clustered” by using the replication feature of Dovecot.
This works well, for most emails.

Sometimes the following happends:

Email to our support database arrives at the inbox.
Every period a cronjob looks into that mailbox and parses the information and 
makes a support ticket from that message.
The cronjob deletes the email afterwards and sees whether there are more new 
mails or not.

In between the deletion and the (next) check, the email gets “resynced” again 
from the “other machine”, and is seen as a new
email by the cronjob and gets handled again. So basically we get two tickets 
for one email. After that second run the email is
no longer replicated.

This behaviour can also be seen via webmail and email clients, this suggests 
that the replication might not be aware (enough) that
the replication is occurring or that an item already had been synced from A to 
B (and does not need to get back from B to A after A
deleted it).

Is there a way to fiddle with the acknowledgement timing or give the processes 
some more space/time to get on par with eachother?

Below is the configuration of machine B, they are synchronised through puppet, 
so only the hostname and IP addresses are different.
(so for replication, A has: tcps:hostname_of_b:12346 and B has 
tcps:hostname_of_a:12346). Also note that HAproxy is prepared but
not in use at all.

# 2.2.33.1 (e9afa7f18): /usr/local/etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.20 (7cd71ba)
# OS: FreeBSD 11.1-RELEASE amd64
auth_mechanisms = plain login
disable_plaintext_auth = no
doveadm_password =  # hidden, use -P to show it
haproxy_trusted_networks = IPv4_Haproxy IPv6_Haproxy
lda_mailbox_autocreate = yes
lda_mailbox_autosubscribe = yes
lmtp_save_to_detail_mailbox = yes
mail_fsync = always
mail_location = sdbox:~/sdbox
mail_plugins = " quota notify replication"
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character 
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy 
include variables body enotify environment mailbox date index ihave duplicate 
mime foreverypart extracttext imapsieve vnd.dovecot.imapsieve
namespace {
  inbox = yes
  location =
  mailbox Drafts {
auto = subscribe
special_use = \Drafts
  }
  mailbox Junk {
special_use = \Junk
  }
  mailbox Sent {
auto = subscribe
special_use = \Sent
  }
  mailbox "Sent Messages" {
special_use = \Sent
  }
  mailbox Spam {
auto = subscribe
special_use = \Junk
  }
  mailbox Trash {
auto = subscribe
special_use = \Trash
  }
  prefix =
  separator = .
}
passdb {
  driver = pam
}
plugin {
  imapsieve_mailbox1_before = 
file:/usr/local/lib/dovecot/sieve/report-spam.sieve
  imapsieve_mailbox1_causes = COPY
  imapsieve_mailbox1_name = Spam
  imapsieve_mailbox2_before = file:/usr/local/lib/dovecot/sieve/report-ham.sieve
  imapsieve_mailbox2_causes = COPY
  imapsieve_mailbox2_from = Spam
  imapsieve_mailbox2_name = *
  mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename
  mail_log_fields = uid box msgid size
  mail_replica = tcps:hostname_of_machine_a:12346
  sieve = ~/.dovecot.sieve
  sieve_after = /usr/local/etc/dovecot/sieve/after/
  sieve_before = /usr/local/etc/dovecot/sieve/global/
  sieve_pipe_bin_dir = /usr/local/lib/dovecot/sieve
  sieve_plugins = sieve_imapsieve sieve_extprograms
  sieve_vacation_dont_check_recipient = yes
}
postmaster_address = postmas...@ourdomain.tld
protocols = imap pop3 lmtp sieve
replication_max_conns = 100
service aggregator {
  fifo_listener replication-notify-fifo {
mode = 0666
  }
  unix_listener replication-notify {
mode = 0666
  }
}
service auth {
  unix_listener /var/spool/postfix/private/auth {
mode = 0666
  }
}
service doveadm {
  inet_listener {
port = 12346
ssl = yes
  }
}
service imap-login {
  inet_listener imap_haproxy {
haproxy = yes
port = 10143
  }
  inet_listener imaps_haproxy {
haproxy = yes
port = 10144
ssl = yes
  }
  service_count = 1
}
service imap {
  process_limit = 1024
}
service lmtp {
  inet_listener lmtp {
address = IPv4, IPv6, 127.0.0.1 ::1
port = 24
  }
  unix_listener /var/spool/postfix/private/dovecot-lmtp {
group = postfix
mode = 0600
user = postfix
  }
}
service pop3 {
  process_limit = 1024
}
service replicator {
  process_min_avail = 1
  unix_listener replicator-doveadm {
mode = 0666
  }
}
ssl_ca = 

signature.asc
Description: Message signed with OpenPGP

Re: is a self signed certificate always invalid the first time?

[Remko Lodder]

Alef,

Certbot creates regular certificates that can be used by dovecot to get a 
“validated” connection to the mailserver.
You obviously need to do the certbot walk to gain the certificate, but if you 
have it, you can use it for dovecot.

Just refer to it in the configuration and you should be fine..

Cheers
Remko

> On 9 Aug 2017, at 17:49, Alef Veld  wrote:
> 
> Thanks Ralph, i’ll look into that.
> 
> I think let’s encrypt uses certbot though and it can’t do email certificates 
> (although i’m sure i can convert the cert i get from let’s encrypt, i’ll look 
> into it.
>> On 9 Aug 2017, at 16:40, Ralph Seichter  wrote:
>> 
>> On 09.08.2017 17:20, Alef Veld wrote:
>> 
>>> So i’m using dovecot, and i created a self signed certificate with
>>> mkcert.sh based on dovecot-openssl.cnf. The name in there matches my
>>> mail server.
>>> 
>>> The first time it connects in mac mail however, it says the certificate
>>> is invalid and another server might pretend to be me etc.
>> 
>> This is to be expected for self-signed certificates. The MUA (Apple Mail
>> in your case) cannot know that the certificate is trusted until you
>> confirm it.
>> 
>> For certificates signed by third parties, the client (or OS) performs
>> the same checks. If a chain of trust can be established based on the
>> client/OS certificate store, which comes pre-populated with well-known
>> third party CA certificates, allowing to verify certificate signatures,
>> your MUA will trust the presented certificate without you confirming it.
>> 
>> I recommend you look into using a free Let's Encrypt certificate (see
>> https://letsencrypt.org/) instead of a self-signed certificate.
>> 
>> -Ralph
> 



signature.asc
Description: Message signed with OpenPGP

Re: pam auth problem

[Remko Lodder]

What is in the pam.d/dovecot file? (Remember to strip passwords if included)

Cheers,

Remko Lodder
 /* sent from my phone and thus brief and to the point *\

Op 3 aug. 2017 om 15:08 heeft Randy Bush  het volgende 
geschreven:

>>> auth: in openpam_parse_chain(): /etc/pam.d/dovecot(1): missing or invalid 
>>> facility
>> 
>> I do not think that it has something to do with the dovecot settings
>> itself but perhaps with the pam facility settings instead?
> 
> i can believe that.  any clues to debug?
> 
> randy

Re: pam auth problem

[Remko Lodder]

Hi Randy,

> On 3 Aug 2017, at 08:50, Randy Bush  wrote:
> 
> auth: in openpam_parse_chain(): /etc/pam.d/dovecot(1): missing or invalid 
> facility

I do not think that it has something to do with the dovecot settings itself but 
perhaps with the pam facility settings instead?

Cheers
Remko


signature.asc
Description: Message signed with OpenPGP

replication issues between to nodes

[Remko Lodder]

Hi,

Some time ago I posted the below but never got a reponse that I could work 
with. So i am retrying now in the hope that there might be a better 
idea/suggestion on how to approach this.

Situation;

I have two nodes, which should replicate to eachother. My main machine receives 
most mail and the other one receives mostly system messages and should get 
replicated. (This used to be delivered on both machines, but given the issues 
below I had to make sure that the customer email at least arrives on machine A, 
as detailed below).

When a mail arrives on main machine (A) everything is fine and things are 
synchronised asap. Customers can see the email directly via webmail/imap.

When a mail arrives on the secondary machine (B) the replication is not issued 
until machine A starts a sync session. Customers do not see the email on 
machine A via webmail/imap.

When a mail arrives on A, the synchronisation occurs, and all messages on B, 
not yet on A, are synchronised as well. Customers can now see the email on 
machine A as well via webmail/imap. Sadly this can mean that emails that became 
visible are hours late (read: were delivered hours before, but not visible for 
the customer).

Both machines are configured through puppet, only individual settings like IP 
addresses and certificates are different because well, they have to. I included 
the difference below, and both ‘doveconf -n’s. If someone has a suggestion on 
seeing why machine B is not issueing (or does not seem to issue) replication, 
let me know. I verified that I can connect to the remote machines via IPv4 and 
IPv6 (for doveadm / replication purposes).

Difference between configurations;

--- tmp1.txt2017-03-18 15:18:41.0 +0100
+++ tmp2.txt2017-03-18 15:18:56.0 +0100
@@ -55,7 +55,7 @@
   imapsieve_mailbox2_name = *
   mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename
   mail_log_fields = uid box msgid size
-  mail_replica = tcps:mail.jr-hosting.nl:12346
+  mail_replica = tcps:mail2.jr-hosting.nl:12346
   sieve = ~/.dovecot.sieve
   sieve_dir = ~/sieve
   sieve_execute_bin_dir = /usr/local/lib/dovecot/sieve
@@ -105,7 +105,7 @@
 }
 service lmtp {
   inet_listener lmtp {
-address = XXX/X 127.0.0.1 ::1
+address = /Y 127.0.0.1 ::1
 port = 24
   }
   unix_listener /var/spool/postfix/private/dovecot-lmtp {
@@ -123,8 +123,8 @@
 mode = 0666
   }
 }
-ssl_ca = 

signature.asc
Description: Message signed with OpenPGP

Re: Master-Master replication question

[Remko Lodder]

Hi Rick,
>> Local delivery on the destination server is LMTP but the transport between
>> MX and destination server is just plain SMTP.
>> 
>> I could try and revert to dovecot-lda and see what that does?
>>  
> I don't think that'll help.  From what I understand, LMTP is required for 
> replication on delivery.
> 
> Out of curiousity, why do you use SMTP from the MX to the destination server 
> instead of LMTP?
> 

It was using that already :-), I do not see a direct reason for this to change, 
I will test it at some point though :)

Re: Master-Master replication question

[Remko Lodder]

> On 11 Jul 2016, at 17:36, Rick Romero  wrote:
> 
> Quoting "William L. Thomson Jr." :
> 
>> You are not alone!
>> 
>> On Wednesday, July 06, 2016 01:15:34 PM Remko Lodder wrote:
>>> Dear list,
>>> 
>>> I have setup a master-master replication setup. My primairy MX's send
>>> email
>>> over on a DNS loadbalanced way, so DNS is doing some kind of round-robin
>>> way of sending mail to both master servers.
>>> 
>>> I found out, that on one of the two machines, the email synchronisation
>>> is
>>> heavily delayed. Lets assume server A receives a mail from the MX; it
>>> synchronises almost instantly with the other server.
>>> 
>>> Whenever server B receives the email, it could take up to several hours
>>> to
>>> synchronise the email, it seems that it is not detected prior.
>> 
>> I have been dealing with this for months.
>> http://www.dovecot.org/list/dovecot/2016-March/103680.html
>> 
>> For a band aid I use  this crontab entry. On the 2nd mail server.
>> 
>> */15 * * * *   root/usr/bin/doveadm sync -u "*" remote:mail1
>> 
> 
> Are you guys using LMTP to deliver from your MX server to the mailbox
> server?

Local delivery on the destination server is LMTP but the transport between
MX and destination server is just plain SMTP.

I could try and revert to dovecot-lda and see what that does?

Cheers
remko


> 
> I have a similar setup, but not yet synched, because as I understand it -
> using 'deliver' to drop mail into an NFS mount won't inititate a sync.  I
> have to migrate my procmail scripts to sieve (and use the execute plugin)
> and change my final delivery to be a redirect to LMTP.   Not sure how
> replication will work when running old procmail scripts from sieve...
> 
> In any case..  If you're piping to dovecot's deliver/dovecot-lda, here is
> a rudimentary LMTP script I hacked together that I planned to use to
> replace deliver with...   I'd grab the 'master' mailbox server IP for
> each user for the command line.
> 
> #!/usr/bin/perl
> 
> use Net::LMTP;
> use Getopt::Std;
> 
> $opts{'s'} = "localhost";
> $opts{'p'} = "24";
> $opts{'f'} = 'root@' . `hostname`;
> chomp($opts{'f'});
> chomp($opts{'s'});
> getopts("hs:p:f:u:", \%opts);
> 
> if ($opts{'h'}) {
> print "
> lmtpsend [-s lmtpserver] [-f fromaddress] [-u subject] toaddress [...]
> 
>   lmtpsend will send an email from the commandline.
> 
>   Options:
> -s lmtpserver  Sets the lmtpserver for where to send the
> mail through.
> -f fromaddress Sets the email address to be used on the
> From: line.
> -u subject Sets the email subject to be used from
> the Subject line.
> toaddress  Where you want the email sent to.
> 
> ";
> exit;
> }
> 
> die "no recepients to send mail to" if ($#ARGV < 0);
> 
> @emailbody =  ;
> 
> # send the message
> 
> $message = Net::LMTP->new($opts{'s'},$opts{'p'}) || die "can't talk to
> server $opts{'s'}\n";
> 
> $message->mail($opts{'f'});
> $message->to(@ARGV) || die "failed to send to the recepients
> ",join(",",@ARGV),": $!";
> $message->data();
> $message->datasend("To: " . join(", ",@ARGV) . "\n");
> $message->datasend(@emailbody);
> $message->dataend();
> $message->quit;
> 
> Rick

Re: Master-Master replication question

[Remko Lodder]

> On 11 Jul 2016, at 17:21, William L. Thomson Jr.  wrote:
> 
> You are not alone!

Hello,

Now that’s a relief!

One of the things that I described and observed is that it seems that serverB 
is not seeing the email (or at least
there is no connection that when an email is send and stored on the mailserver 
that the services see them and
notify the other end). With tcpdump there is no traffic at all, until there is 
a sync the other way around.

As said both systems are identical in hardware setup and use puppet to obtain 
their configuration, which is the same
for both hosts (except the IP adresses and hostname);

But since we are with at least two, we might have better luck in getting some 
help with this. I currently do not have
an idea on where to look and how to investigate this properly.

Any pointers from the list are welcome!

Cheers
Remko

> 
> -- 
> William L. Thomson Jr.
> Obsidian-Studios, Inc.
> http://www.obsidian-studios.com

Master-Master replication question

[Remko Lodder]

Dear list,

I have setup a master-master replication setup. My primairy MX's send email 
over on a DNS loadbalanced way, so DNS is doing some kind of round-robin way of 
sending mail to both master servers.

I found out, that on one of the two machines, the email synchronisation is 
heavily delayed.
Lets assume server A receives a mail from the MX; it synchronises almost 
instantly with the other server.

Whenever server B receives the email, it could take up to several hours to 
synchronise the email, it seems that it is not detected prior.

It is also interesting to see, that the mailboxes on server A (Where users 
login to retrieve their email via webmail/clients) are significantly smaller 
then the mailboxes on server B. When investigating, it seems that "older" 
mailboxes (or storage rather since we use mdbox) are still there on server B, 
which already had been removed on server A.

My personal mailbox was 170MB on server A, while it was still 2.5GB on server 
B. (which was around that size before cleaning up the mailsboxes).

I enabled debugging on the servers, and I see rather quick : "Replication 
requests" on server A, but when getting an email on server B, I do not see the 
request at all.

My servers are both running the same version, same configuration (utilizing 
puppet), both running on ZFS and FreeBSD. Where server B is more loaded in it's 
memory because of some bhyve VM's and the server A does not run any VM.

Does someone have any pointers on where to look?

Thanks in advance ;-)
Remko

Included below the configurations from server A and B:

Server A:
# 2.2.25 (7be1766): /usr/local/etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.14 (099a97c)
# OS: FreeBSD 10.3-RELEASE-p2 amd64  
auth_mechanisms = plain login
disable_plaintext_auth = no
doveadm_password =  # hidden, use -P to show it
haproxy_trusted_networks = 
lda_mailbox_autocreate = yes
lda_mailbox_autosubscribe = yes
lmtp_save_to_detail_mailbox = yes
mail_debug = yes
mail_fsync = always
mail_location = mdbox:~/mdbox
mail_plugins = " quota notify replication"
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character 
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy 
include variables body enotify environment mailbox date index ihave duplicate 
mime foreverypart extracttext
namespace {
  inbox = yes
  location = 
  mailbox Drafts {
auto = subscribe
special_use = \Drafts
  }
  mailbox Junk {
special_use = \Junk
  }
  mailbox Sent {
auto = subscribe
special_use = \Sent
  }
  mailbox "Sent Messages" {
special_use = \Sent
  }
  mailbox Spam {
auto = subscribe
special_use = \Junk
  }
  mailbox Trash {
auto = subscribe
special_use = \Trash
  }
  prefix = 
  separator = .
}
passdb {
  driver = pam
}
plugin {
  antispam_backend = mailtrain
  antispam_mail_notspam = --ham
  antispam_mail_sendmail = /usr/local/bin/sa-learn.sh
  antispam_mail_spam = --spam
  antispam_spam_pattern_ignorecase = spam;junk
  antispam_trash_pattern_ignorecase = trash;deleted items;deleted messages
  antispam_verbose_debug = 1
  mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename
  mail_log_fields = uid box msgid size
  mail_replica = tcps:the other server:12346
  sieve = ~/.dovecot.sieve
  sieve_dir = ~/sieve
  sieve_global_dir = /usr/local/etc/dovecot/sieve/global/
  sieve_global_path = /usr/local/etc/dovecot/sieve/default.sieve
}
postmaster_address = postmaster@xxx
protocols = imap pop3 lmtp sieve
replication_dsync_parameters = -d -N -l 60 -U
replication_max_conns = 100
service aggregator {
  fifo_listener replication-notify-fifo {
mode = 0666
  }
  unix_listener replication-notify {
mode = 0666
  }
}
service auth {
  unix_listener /var/spool/postfix/private/auth {
mode = 0666
  }
}
service doveadm {
  inet_listener {
port = 12346
ssl = yes
  }
}
service imap-login {
  inet_listener imap_haproxy {
haproxy = yes
port = 10143
  }
  inet_listener imaps_haproxy {
haproxy = yes
port = 10144
ssl = yes
  }
  service_count = 1
}
service imap {
  process_limit = 1024
}
service lmtp {
  unix_listener /var/spool/postfix/private/dovecot-lmtp {
group = postfix
mode = 0600
user = postfix
  }
}
service pop3 {
  process_limit = 1024
}
service replicator {
  process_min_avail = 1
  unix_listener replicator-doveadm {
mode = 0666
  }
}
ssl_ca = 

Re: FreeBSD 10 & default_vsz_limit causing reboots?

[Remko Lodder]

Hi,

> Op 15 september 2015 om 14:52 schreef Rick Romero :
> 
> 
> Ok,
> 
> So this is really more of an observation than anything else. 
> 
> I had a FreeBSD 10.1 server that was running great. Some SSL issue came up,
> or I upgrade Dovecot in ports - something occurred and the machine started
> rebooting randomly.  It would run for 2 weeks, then reboot.  It might run
> for 5 days and then reboot. So I started doing more FreeBSD upgrades,
> thinking it was a kernel issue. The reboots only increased. 
> 
> This weekend I started thinking I might actually be having hardware
> issues.  But, since I don't have easy physical access to the box and it's
> REALLY under loaded, I figured what the hell and upraded to 10.2 on
> Sunday.  I think it rebooted 4 times after that on Sunday, and then
> another 2 times Monday morning. 
> 
> Its worth noting that while I have crash dumps enabled, they don't seem to
> be occurring.  So hardware is still a possibility.

Jumping in at some point here, as FreeBSD dev I run most of my servers on
FreeBSD. All my mailservers are running FreeBSD.
My customer backend servers run Dovecot on FreeBSD. we have a few hundred
mailboxes (not that many). I upgrade all my packages
and system whenever there are updates and I figured out whether they are OK or
not. That means that I most likely do more upgrades
then you do at the moment.

I never ever had the symptoms you describe nor did I need to tweak settings.

Given this is a "FreeBSD"box crashing I thought I should reply. I think you need
to contact the FreeBSD devs (other then me) to ask what is going on.
Perhaps you can do a backtrace on the dump to see what was going on. If you
installed panicmail (a tool by Colin Percival) it will automatically create
an informative email which describes the issue more or less ..

Please poke me offline when I can help you more with that.

Cheers
Remko(@FreeBSD.org)

> 
> After the 2nd Monday morning reboot, I started to wonder if there was some
> sort of process issue.  Besides the OS upgrades - I had been monitoring
> the Dovecot logs for when the process limits are reached, and increasing
> them.  It's a 'big' box, and load is typically between .30 and .50. CPUs
> aren't overtaxed, and most of the memory is dedicated to ZFS.  The reboots
> are so short, I've only received one 'down' alert due to them. So it's a
> conerning issue, but not really impacting production.
> 
> On a whim I changed my default_vsz_limit (as I had been increasing every
> other limit but that) from 384M to 512M.  The system hasn't rebooted in
> 24hours.
> 
> Now that could be a coincidence, but I thought I'd at least put it out
> there.
> 
> If you see anything weird in my dovecot config, let me know - My config was
> originally vpopmail, but over time I've migrated to SQL-only.
> 
> root@romulus:/usr/local/etc/dovecot # dovecot -n
> # 2.2.18: /usr/local/etc/dovecot/dovecot.conf
> # OS: FreeBSD 10.2-RELEASE amd64
> auth_master_user_separator = *
> auth_mechanisms = plain login
> auth_username_translation = %@
> auth_verbose = yes
> default_login_user = dovecot
> default_vsz_limit = 512 M
> disable_plaintext_auth = no
> first_valid_gid = 89
> first_valid_uid = 89
> last_valid_gid = 89
> last_valid_uid = 89
> log_path = /dev/stderr
> login_greeting = Ready.
> login_trusted_networks = 172.16.100.0/24
> mail_fsync = never
> mail_plugins = " quota zlib stats"
> mail_privileged_group = mail
> namespace compat {
>   alias_for =
>   hidden = yes
>   inbox = no
>   list = no
>   location =
>   prefix = INBOX.
>   separator = .
> }
> namespace inbox {
>   inbox = yes
>   location =
>   prefix =
>   separator = .
> }
> passdb {
>   args = /usr/local/etc/dovecot/dovecot-master-sql.conf
>   driver = sql
>   master = yes
>   pass = yes
> }
> passdb {
>   args = /usr/local/etc/dovecot/dovecot-sql.conf
>   driver = sql
> }
> plugin {
>   quota = maildir
>   quota_rule = Trash:storage=+10%%
>   stats_refresh = 30 secs
>   stats_track_cmds = yes
> }
> protocols = imap pop3
> service anvil {
>   client_limit = 3175
> }
> service auth {
>   client_limit = 3684
>   unix_listener auth-master {
>     mode = 0600
>   }
> }
> service imap-login {
>   process_limit = 1536
>   process_min_avail = 25
>   service_count = 1
> }
> service imap-postlogin {
>   executable = script-login rawlog /usr/local/etc/dovecot/lastauth-imap.sh
>   user = vpopmail
> }
> service imap {
>   executable = /usr/local/libexec/dovecot/imap imap-postlogin
>   process_limit = 1536
> }
> service pop-postlogin {
>   executable = script-login /usr/local/etc/dovecot/lastauth-pop.sh
>   user = vpopmail
> }
> service pop3-login {
>   process_limit = 1536
>   process_min_avail = 15
>   service_count = 1
> }
> service pop3 {
>   executable = /usr/local/libexec/dovecot/pop3 pop-postlogin
> }
> service stats {
>   fifo_listener stats-mail {
>     mode = 0600
>     user = vpopmail
>   }
> }
> shutdown_clients = no
> ssl_cert =  ssl_key =  ssl_key_password = na
> userdb {
>   driver = pref

Re: How to "Windows Authenticate"

[Remko Lodder]

> On 16 Sep 2015, at 19:10, Mark Foley  wrote:
> 
> Does the Dovecot NTLM mechanism work with MS Outlook?
> 
> [ ] YES
> [ ] NO
> 
> Please check one ... anybody.
> 
> --Mark
> 
> 

[checking not suited for work]:

: host mail.ohprs.org[98.102.63.107] said: 550 5.7.1 Access
   denied (in reply to MAIL FROM command)

You are welcome :-p


--
/"\   Best regards,          | re...@freebsd.org
\ /   Remko Lodder   | remko@EFnet
 Xhttp://www.evilcoder.org/  |
/ \   ASCII Ribbon Campaign  | Against HTML Mail and News



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: How to "Windows Authenticate"

[Remko Lodder]

> On 16 Sep 2015, at 19:10, Mark Foley  wrote:
> 
> Does the Dovecot NTLM mechanism work with MS Outlook?
> 
> [ ] YES
> [ ] NO
> 
> Please check one ... anybody.
> 
> —Mark



The URL on the wiki, which had probably been shared before with you;

http://wiki2.dovecot.org/HowTo/ActiveDirectoryNtlm

suggests it does.

The URL quotes:

Step 5. Passwordless authentication

If you have logged on from Windows to the AD domain, try leaving the password 
field, on the account, on the MUA, blank. The username / password, from the 
initial logon to the Windows machine, are seamlessly picked up and supplied to 
the challenge-response process between the MUA, Dovecot and AD. Employing this 
way of authentication  we achieve single sign-on and we don't need to maintain 
MUA local passwords.

Did you follow the suggestions that are on that page? (all of them).

Thank you,
Remko

--
/"\   Best regards,      | re...@freebsd.org
\ /   Remko Lodder   | remko@EFnet
 Xhttp://www.evilcoder.org/  |
/ \   ASCII Ribbon Campaign  | Against HTML Mail and News



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: Question wrt. dovecot replicator

[Remko Lodder]

> On 30 Oct 2014, at 17:31, Jan Behrend  wrote:
> 
> On Fri, 2014-10-10 at 14:45 +0200, Jan Behrend wrote:
>> On Fri, 2014-10-03 at 23:37 +0200, Remko Lodder wrote:
>>>>>> How can I determine why there are duplicated emails?
>> 
>> Same problem here!
>> 
>>>>>> What kind of messages should I specifically look for?
>>>>> Look for any errors and warnings in the Dovecot log. You could also 
>>>>> enable mail_debug (ref.
>> 
>> 
>>>>>> Can I set this up for a few selected accounts instead of all accounts 
>>>>>> like it was currently? To make sure I do not make things worse for 
>>>>>> others then needs to be :-)
>>>>>> The service had been disabled for the time being to prevent the other 
>>>>>> users from getting duplicated emails.
>>>>> I do not know what kind of userdb you are running, but there is a newish 
>>>>> patch that enables per user replication via the
>>>>> mail_replica setting. It is not yet included in the newest (2.2.13) 
>>>>> release of Dovecot, but is available via the enterprise
>>>>> version. There are no FreeBSD builds for that, though. ref: 
>>>>> http://hg.dovecot.org/dovecot-2.2/rev/c1c67bdc8752
>>>> 
>>>> my userdb consists of local users (Which are fed through LDAP at the 
>>>> backend). perhaps I can setup a mailAttributes setting or something so 
>>>> that the replica can be
>>>> set, although I prefer that I have control over that in the config itself 
>>>> :-)
>> 
>> With the latest Debian jessie version 1:2.2.13-5 you can actually have a
>> per user mail_replica setting taken from a (LDAP) directory.  This keeps
>> the duplicate mail issue away from other users but a few willing to
>> test ...
>> 
>>> For what it’s worth:
>>> 
>>> replication_dsync_parameters = -f -d -N -l 30 -U
>> 
>> I read in Peer Heinlein's Dovecot book
>> http://www.opensourcepress.de/de/produkte/Dovecot/13560/978-3-95539-074-7
>> that replicating a public namespace gives you trouble :-(. So keep the
>> "-N" option away for now.
>> 
>> However I would like examples for the "-n" and "-x" options, which are
>> neither given in the wiki nor in the nonexistant man page.
>> 
>> I think the replication feature is very, very cool, but right now it
>> gives me a hard time to implement flawlessly ;-)
>> 
>> Thanks for any help or light shed on this issue ...
> 
> Found it ;-)
> 
> http://wiki2.dovecot.org/Tools/Doveadm/Sync
> 
> All working beautifully now!

For what it’s worth it is working fine here with debugging enabled. I will 
upgrade to 2.2.15 and keep testing that to see how that works.

Thanks for the help and suggestions so far!

Remko

> 
> Cheers Jan
> 
> -- 
> MAX-PLANCK-INSTITUT fuer Radioastronomie
> Jan Behrend - Rechenzentrum
> 
> Auf dem Huegel 69, D-53121 Bonn  
> Tel: +49 (228) 525 359, Fax: +49 (228) 525 229
> jbehr...@mpifr-bonn.mpg.de http://www.mpifr-bonn.mpg.de
> 
> 

Re: Question wrt. dovecot replicator

[Remko Lodder]

On 03 Oct 2014, at 23:27, Remko Lodder  wrote:

> 
> On 02 Oct 2014, at 10:57, Teemu Huovila  wrote:
> 
>> On 10/02/2014 02:40 AM, Remko Lodder wrote:
>>> and a mail_replica = tcp:host{a,b}:12346 configuration on each host so that 
>>> they are pointing to eachother; This seems to work fine for most accounts, 
>>> for example: I never experienced issues with this. However, several other 
>>> accounts (with a large variety of clients) got duplicated emails. Looking 
>>> with doveadm I only noticed that the numbers of the messages are closely 
>>> related to eachother but one number incremented. So they cannot be deleted 
>>> with the deduplicator function.
>>> 
>>> The replication is provided over TCP only, the connection streams over an 
>>> OpenVPN tunnel so that the contents are protected, the machines are located 
>>> in different Datacenters but close to eachother.
>>> 
>>> How can I determine why there are duplicated emails?
>>> What kind of messages should I specifically look for?
>> Look for any errors and warnings in the Dovecot log. You could also enable 
>> mail_debug (ref.
>> http://wiki2.dovecot.org/Logging#Logging_verbosity) for the accounts being 
>> synced. Also, please post your complete configuration.
> 
> I will set that over the weekend with the replication settings re-enabled. I 
> will also post the configuration with the hopefully gathered debugging 
> information.
> 
>> 
>>> Can I set this up for a few selected accounts instead of all accounts like 
>>> it was currently? To make sure I do not make things worse for others then 
>>> needs to be :-)
>>> The service had been disabled for the time being to prevent the other users 
>>> from getting duplicated emails.
>> I do not know what kind of userdb you are running, but there is a newish 
>> patch that enables per user replication via the
>> mail_replica setting. It is not yet included in the newest (2.2.13) release 
>> of Dovecot, but is available via the enterprise
>> version. There are no FreeBSD builds for that, though. ref: 
>> http://hg.dovecot.org/dovecot-2.2/rev/c1c67bdc8752
> 
> my userdb consists of local users (Which are fed through LDAP at the 
> backend). perhaps I can setup a mailAttributes setting or something so that 
> the replica can be
> set, although I prefer that I have control over that in the config itself :-)
> 
> Thanks!
> Remko

For what it’s worth:

# 2.2.13: /usr/local/etc/dovecot/dovecot.conf
# OS: FreeBSD 10.0-RELEASE-p7 amd64  
auth_mechanisms = plain login
disable_plaintext_auth = no
doveadm_password = XX
mail_debug = yes
mail_location = maildir:~/Maildir
mail_plugins = " quota notify"
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character 
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy 
include variables body enotify environment mailbox date ihave duplicate
passdb {
  driver = pam
}
plugin {
  antispam_backend = mailtrain
  antispam_mail_notspam = --ham
  antispam_mail_sendmail = /usr/local/bin/sa-learn.sh
  antispam_mail_spam = --spam
  antispam_spam = Spam
  antispam_trash = trash;Trash;Deleted Items; Deleted Messages; .Trash
  mail_replica = tcp::12346
  sieve = ~/.dovecot.sieve
  sieve_dir = ~/sieve
  sieve_global_dir = /usr/local/etc/dovecot/sieve/global/
  sieve_global_path = /usr/local/etc/dovecot/sieve/default.sieve
}
postmaster_address = postmaster@
protocols = imap pop3 lmtp sieve
replication_dsync_parameters = -f -d -N -l 30 -U
replication_max_conns = 30
service aggregator {
  fifo_listener replication-notify-fifo {
mode = 0666
  }
  unix_listener replication-notify {
mode = 0666
  }
}
service auth {
  unix_listener /var/spool/postfix/private/auth {
mode = 0666
  }
}
service doveadm {
  inet_listener {
port = 12346
  }
}
service imap-login {
  service_count = 1
}
service imap {
  process_limit = 1024
}
service lmtp {
  unix_listener /var/spool/postfix/private/dovecot-lmtp {
group = postfix
mode = 0600
user = postfix
  }
}
service pop3 {
  process_limit = 1024
}
service replicator {
  process_min_avail = 1
  unix_listener replicator-doveadm {
mode = 0666
  }
}
ssl_ca = < 
ssl_cert = < 
ssl_key = < 
userdb {
  driver = passwd
}
protocol lmtp {
  mail_plugins = quota sieve
  postmaster_address = postmaster@
}
protocol lda {
  mail_plugins = " quota notify sieve"
}
protocol imap {
  mail_max_userip_connections = 20
  mail_plugins = " quota notify imap_quota antispam"
}
[ro

> 
>> 
>> br,
>> Teemu Huovila
> 
> -- 
> 
> /"\   Best regards,  | re...@freebsd.org

Re: Question wrt. dovecot replicator

[Remko Lodder]

On 02 Oct 2014, at 10:57, Teemu Huovila  wrote:

> On 10/02/2014 02:40 AM, Remko Lodder wrote:
>> and a mail_replica = tcp:host{a,b}:12346 configuration on each host so that 
>> they are pointing to eachother; This seems to work fine for most accounts, 
>> for example: I never experienced issues with this. However, several other 
>> accounts (with a large variety of clients) got duplicated emails. Looking 
>> with doveadm I only noticed that the numbers of the messages are closely 
>> related to eachother but one number incremented. So they cannot be deleted 
>> with the deduplicator function.
>> 
>> The replication is provided over TCP only, the connection streams over an 
>> OpenVPN tunnel so that the contents are protected, the machines are located 
>> in different Datacenters but close to eachother.
>> 
>> How can I determine why there are duplicated emails?
>> What kind of messages should I specifically look for?
> Look for any errors and warnings in the Dovecot log. You could also enable 
> mail_debug (ref.
> http://wiki2.dovecot.org/Logging#Logging_verbosity) for the accounts being 
> synced. Also, please post your complete configuration.

I will set that over the weekend with the replication settings re-enabled. I 
will also post the configuration with the hopefully gathered debugging 
information.

> 
>> Can I set this up for a few selected accounts instead of all accounts like 
>> it was currently? To make sure I do not make things worse for others then 
>> needs to be :-)
>> The service had been disabled for the time being to prevent the other users 
>> from getting duplicated emails.
> I do not know what kind of userdb you are running, but there is a newish 
> patch that enables per user replication via the
> mail_replica setting. It is not yet included in the newest (2.2.13) release 
> of Dovecot, but is available via the enterprise
> version. There are no FreeBSD builds for that, though. ref: 
> http://hg.dovecot.org/dovecot-2.2/rev/c1c67bdc8752

my userdb consists of local users (Which are fed through LDAP at the backend). 
perhaps I can setup a mailAttributes setting or something so that the replica 
can be
set, although I prefer that I have control over that in the config itself :-)

Thanks!
Remko

> 
> br,
> Teemu Huovila

-- 

/"\   Best regards,  | re...@freebsd.org
\ /   Remko Lodder   | remko@EFnet
 Xhttp://www.evilcoder.org/  |
/ \   ASCII Ribbon Campaign  | Against HTML Mail and News



signature.asc
Description: Message signed with OpenPGP using GPGMail

Question wrt. dovecot replicator

[Remko Lodder]

Hi People,

I recently setup a replication service within dovecot between two machines in 
different datacenters, which has the following configuration;
(yes it is hashed out, because I needed to)

#service replicator {
#  unix_listener replicator-doveadm {
#mode = 0666
#  }
#  process_min_avail = 1
#}
#
#service aggregator {
#  fifo_listener replication-notify-fifo {
#mode = 0666
##user = vmail
#  }
#  unix_listener replication-notify {
#mode = 0666
##user = vmail
#  }
#}
#
#service doveadm {
#  inet_listener {
#port = 12346
#  }
#}
#
##doveadm_port = 12346
#doveadm_password = XXX
#replication_max_conns = 30
#

and a mail_replica = tcp:host{a,b}:12346 configuration on each host so that 
they are pointing to eachother; This seems to work fine for most accounts, for 
example: I never experienced issues with this. However, several other accounts 
(with a large variety of clients) got duplicated emails. Looking with doveadm I 
only noticed that the numbers of the messages are closely related to eachother 
but one number incremented. So they cannot be deleted with the deduplicator 
function.

The replication is provided over TCP only, the connection streams over an 
OpenVPN tunnel so that the contents are protected, the machines are located in 
different Datacenters but close to eachother.

How can I determine why there are duplicated emails?
What kind of messages should I specifically look for?

Can I set this up for a few selected accounts instead of all accounts like it 
was currently? To make sure I do not make things worse for others then needs to 
be :-)
The service had been disabled for the time being to prevent the other users 
from getting duplicated emails.

Thanks for the advice in advance!

Cheers
Remko


-- 

/"\   Best regards,  | re...@freebsd.org
\ /   Remko Lodder   | remko@EFnet
 Xhttp://www.evilcoder.org/  |
/ \   ASCII Ribbon Campaign  | Against HTML Mail and News



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: Does dovecot work OK on *BSD?

[Remko Lodder]

On 25 Sep 2014, at 18:14, Steve Litt  wrote:

> Hi all,
> 
> I have a dovecot server on my Debian Wheezy desktop computer. My days
> with Debian are limited, and I'm investigating several 'BSD's:
> 
> OpenBSD
> FreeBSD
> PCBSD
> NetBSD
> DragonflyBSD
> etc
> 
> Is there any reason Dovecot wouldn't work on any of those? Does anyone
> know if those OS's have packages for Dovecot, or do I need to compile
> it myself?
> 
> Thanks,
> 
> SteveT

it works just fine for my little company, see my signature on what OS I prefer.
Poke me in case you need help ..

Cheerio
Remko

> 
> Steve Litt*  http://www.troubleshooters.com/
> Troubleshooting Training  *  Human Performance

-- 

/"\   Best regards,  | re...@freebsd.org
\ /   Remko Lodder   | remko@EFnet
 Xhttp://www.evilcoder.org/  |
/ \   ASCII Ribbon Campaign  | Against HTML Mail and News



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: LDAP and dovecot

[Remko Lodder]

On 19 Sep 2014, at 23:31, dove...@outputservices.com wrote:

> I am sorry. 
> 
> I am NOW able to authenticate to my ldap server.
> 
> Thank you all.

So what changed, for the records :)

Remko

-- 

/"\   Best regards,  | re...@freebsd.org
\ /   Remko Lodder   | remko@EFnet
 Xhttp://www.evilcoder.org/  |
/ \   ASCII Ribbon Campaign  | Against HTML Mail and News



signature.asc
Description: Message signed with OpenPGP using GPGMail