Re: Merging existing mailboxes to aliases

2019-05-19 Thread Sami Ketola via dovecot



> On 19 May 2019, at 7.08, Lefteris Tsintjelis via dovecot 
>  wrote:
> 
> I am trying to merge all, existing or not, different mailboxes in one with 
> the mailbox alias plugin. If I add the following lines, will the existing 
> mailboxes automatically merge in one mailbox and corresponding links auto 
> create? If not, do I have to do this manually?
> 
>  mailbox_alias_new = Spam
>  mailbox_alias_new2 = Junk E-mail
>  mailbox_alias_new3 = Sent Items
>  mailbox_alias_new4 = Sent Messages
>  mailbox_alias_new5 = Deleted Items
>  mailbox_alias_old = Junk
>  mailbox_alias_old2 = Junk
>  mailbox_alias_old3 = Sent
>  mailbox_alias_old4 = Sent
>  mailbox_alias_old5 = Trash


Why do you want to do this? Thunderbird does obey special_use flags.  
https://wiki.dovecot.org/MailboxSettings

Do you have other clients then that do not honor those?

Sami

Re: doveadm sync issues (revised)

2019-05-09 Thread Sami Ketola via dovecot



> On 9 May 2019, at 23.47, Sven Strickroth via dovecot  
> wrote:
> 
> Hi,
> 
> I'm experiencing serious issues using doveadm sync; mails are not
> replicating and/or mails are duplicating. I wanted to use this for
> migrating external mailboxes to my server (in order to overcome the time
> when some servers/clients still use the old server and some already my
> server, as described on ).

Use proxy for that. 

> 
> Now send two messages to rem...@domain.tld (Subjects Test3 and Test4)
> and two messages to lo...@domain.tld (Subjects Test5 and Test6).
> 
> Run sync:
> $ doveadm -o imapc_user="rem...@domain.tld" -o imapc_password="PW" -o
> imapc_host="localhost" sync -R -u lo...@domain.tld imapc:

imapc does not support two-way sync. You need to use doveadm protocol for that.

Sami

Re: Feature request: exclude IP/network in allow_nets extra field

2019-04-30 Thread Sami Ketola via dovecot



> On 30 Apr 2019, at 4.56, Zhang Huangbin via dovecot  
> wrote:
> 
> Dear all,
> 
> We use `allow_nets`[1] to restrict login clients, it works fine.
> Recently we need to allow some users to login from everywhere except some 
> IP/networks, how can we accomplish this with "allow_nets"?
> 
> Tried allow_nets="!a.b.c.d", but Dovecot reports error "allow_nets: Invalid 
> network '!a.b.c.d'".
> 
> Can we have this feature?


Just create another passdb for these premium users before the actual passdb and 
add skip = authenticated to the actual passdb.

Sami



Re: Extended logging / moved mails jumping back

2019-04-19 Thread Sami Ketola via dovecot


> On 19 Apr 2019, at 20.04, Martin Müller via dovecot  
> wrote:
> 
> 2019-04-19 18:53:54 imap-login: Info: Login: user= >, method=PLAIN, rip=80.75.xx.35, 
> lip=136.xxx.9.172, mpid=28364, TLS, session=
> 
> All 4 different MUAs Thunderbird are logged in the same way. They are behind 
> a router, so they having the same remote IP. 
> So I cant differentiate, which MUA causes which event.
> 
> Is there a way, to identify which client raise a special event?

I think Thunderbird does send IMAP ID information so you can try adding 
"imap_id_log = *" to your config to get the info logged.

Maybe it has some information to identify the different clients.

Sami



Re: Lua Push Notification Plugin

2019-04-11 Thread Sami Ketola via dovecot

Hi,

doveadm mailbox metadata get -u victim INBOX 
/private/vendor/vendor.dovecot/pvt/server/vendor/vendor.dovecot/http-notify

or doveadm mailbox metadata set

if you are not using appsuite as your mail frontend then you need to set the 
metadata manually or make sure that your mail frontend does set it.

Sami

> On 11 Apr 2019, at 11.50, Pabsky  wrote:
> 
> Sami,
> 
> the package 'appsuite' is not mentioned in the documentation, also I already 
> enabled imap metadata. 
> 
> What do you mean by 'It's not supposed to be a file. It's supposed to be a 
> attribute on the users INBOX'?
> 
> I'm attaching my dovecot configuration file for you to examine. Thanks Sami!
> 
> On 11/04/2019 4:37 PM, Sami Ketola wrote:
>> 
>> 
>>> On 11 Apr 2019, at 11.00, Pabsky via dovecot >> <mailto:dovecot@dovecot.org>> wrote:
>>> 
>>> Thanks AKI! I'm a step closer to achieving my goals. 
>>> 
>>> However, I'm getting a new error as indicated from below:
>>> 
>>> Apr 11 01:45:34 lmtp(u...@mydomain.com 
>>> <mailto:u...@mydomain.com>)<20801>: Debug: 
>>> push-notification-ox: Skipped because not active 
>>> (/private/vendor/vendor.dovecot/pvt/server/vendor/vendor.dovecot/http-notify
>>>  METADATA not set)
>>> Apr 11 01:45:34 lmtp(u...@mydomain.com 
>>> <mailto:u...@mydomain.com>)<20801>: Debug: 
>>> push-notification: Push notification transaction completed
>>> 
>>> By the way, the file 
>>> /private/vendor/vendor.dovecot/pvt/server/vendor/vendor.dovecot/http-notify 
>>> does not exist on my server.
>> 
>> It's not supposed to be a file. It's supposed to be a attribute on the users 
>> INBOX. Also you need to enable imap metadata or appsuite can't set the 
>> attribute.
>> 
>> Sami
>> 
>> 
> 



Re: Lua Push Notification Plugin

2019-04-11 Thread Sami Ketola via dovecot


> On 11 Apr 2019, at 11.00, Pabsky via dovecot  wrote:
> 
> Thanks AKI! I'm a step closer to achieving my goals. 
> 
> However, I'm getting a new error as indicated from below:
> 
> Apr 11 01:45:34 lmtp(u...@mydomain.com 
> )<20801>: Debug: 
> push-notification-ox: Skipped because not active 
> (/private/vendor/vendor.dovecot/pvt/server/vendor/vendor.dovecot/http-notify 
> METADATA not set)
> Apr 11 01:45:34 lmtp(u...@mydomain.com 
> )<20801>: Debug: 
> push-notification: Push notification transaction completed
> 
> By the way, the file 
> /private/vendor/vendor.dovecot/pvt/server/vendor/vendor.dovecot/http-notify 
> does not exist on my server.

It's not supposed to be a file. It's supposed to be a attribute on the users 
INBOX. Also you need to enable imap metadata or appsuite can't set the 
attribute.

Sami




Re: doveadm backup + ISO 8859-1 mailbox name

2019-04-04 Thread Sami Ketola via dovecot


> On 4 Apr 2019, at 11.03, Michael Goth via dovecot  wrote:
> 
> That too includes the  byte in its output. But I've found that only a 
> couple of mailboxes are affected, so i just renamed the folders.
> 
> Thanks for your help!


The legacy server is then not RFC 3501 compliant. 8-bit folder names should be 
encoded in mUTF7.

But if you already solved the issue by renaming folders then I guess you are 
already fine.

Sami

Re: doveadm backup + ISO 8859-1 mailbox name

2019-04-03 Thread Sami Ketola via dovecot



> On 3 Apr 2019, at 13.40, Michael Goth via dovecot  wrote:
> 
> Hi!
> 
> I'm trying to migrate a mailbox from an old IMAP server (not Dovecot) to a 
> new Dovecot server with 'doveadm backup'. The command fails because of an 
> unexpected character in a mailbox name:
> 
> 
> doveadm -o imapc_user=a...@example.org \
>-o imapc_password=XXX \
>-o imapc_host=old-mailserver.webflow.de \
>-o imapc_port=993 \
>-o imapc_ssl=imaps \
>-o imapc_ssl_verify=no \
>-o imapc_features=rfc822.size,fetch-headers \
>-o ssl_client_ca_dir=/etc/ssl \
>backup -R -f -x ~* -u migration...@example.org imapc:
> 
> dsync(migration...@example.org): Info: imapc(old-mailserver.webflow.de:993): 
> Connected to 1.1.1.1:993 (local 2.2.2.2:46154)
> dsync(migration...@example.org): Error: Failed to access mailbox Entw?rfe: 
> mailbox does not exist
> 
> 
> The problem is with "Entw?rfe" here. It's actually Entw<0xfc>rfe, which 
> translates to "Entwürfe" in ISO-8859-1.
> 
> Why can't doveadm find that mailbox? Is the old mailserver misbehaving (I 
> hope not because I have no idea how to change that...), or does this happen 
> on the new server? Can I tell doveadm something that makes this work?
> 

Can you log on the remote imap server with telnet or nc and ask for: A LIST "" 
"*"

I think your legacy server does not encode the folder names correctly.

Sami



Re: CentOS Repository broken ?

2019-03-31 Thread Sami Ketola via dovecot
Hi,

remove package yum-plugin-priorities. I think it installs by default now and 
makes yum to prioritise base repo over any other repo.

or.. can also configure the plugin to prioritise dovecot repo over base.

Sami


> On 31 Mar 2019, at 16.01, Günther J. Niederwimmer via dovecot 
>  wrote:
> 
> Hello,
> 
> I download and install it manual, this working now?
> 
> waiting for next update to found out, yum is working correct again ;-)
> 
> Thanks for answers,
> 
> Am Sonntag, 31. März 2019, 13:59:17 CEST schrieb Kostya Vasilyev via dovecot:
>> On Sun, Mar 31, 2019, at 2:30 PM, Günther J. Niederwimmer via dovecot wrote:
>>> Hello,
>>> 
>>> Am Samstag, 30. März 2019, 21:36:12 CEST schrieb Peter via dovecot:
 On 30/03/19 10:56 PM, Gerald Galster via dovecot wrote:
> Maybe there is something wrong with his local yum/repo configuration.
 
 I'm tending to agree now.
>>> 
>>> [ ... ]
>>> 
 It would have to be for yum to see dovecot-imaptest-2.3.5.1-1.x86_64 and
 dovecot-lua-2.3.5.1-1.x86_64.  It sounds more like he's put an
 exclude=dovecot line in his dovecot-2.3-latest repo container.
>>> 
>>> NO, I have no exclude in the dovecot.repo but it is not possible for yum
>>> to
>>> find the dovecot Package but the dovecot-lua ... is found?
>>> 
>>> I mean there are wrong permissions on the repo?
>> 
>> FWIW, works for me just fine.
>> 
>> I created a brand new, fresh, CentOS 7 image in the cloud.
>> 
>> Configured Dovecot repo as described here:
>> 
>> https://repo.dovecot.org/
>> 
>> Did yum install and it worked just fine (see the output below).
>> 
>> Maybe you can do
>> 
>> yum list "dovecot*"
>> 
>> 
>> 
>> So we can see what you have installed right now vs. what your system sees in
>> the repo?
>> 
>> -
>> 
>> # yum install dovecot dovecot-lua dovecot-imaptest
>> 
>> Resolving Dependencies
>> --> Running transaction check
>> ---> Package dovecot.x86_64 2:2.3.5.1-1 will be installed
>> --> Processing Dependency: libclucene-core.so.1()(64bit) for package:
>> 2:dovecot-2.3.5.1-1.x86_64 --> Processing Dependency:
>> libclucene-shared.so.1()(64bit) for package: 2:dovecot-2.3.5.1-1.x86_64
>> ---> Package dovecot-imaptest.x86_64 2:2.3.5.1-1 will be installed --->
>> Package dovecot-lua.x86_64 2:2.3.5.1-1 will be installed
>> --> Running transaction check
>> ---> Package clucene-core.x86_64 0:2.3.3.4-11.el7 will be installed
>> --> Finished Dependency Resolution
>> 
>> Dependencies Resolved
>> 
>> 
>> == Package Arch 
>>VersionRepository Size
>> ===
>> === Installing:
>> dovecot x86_642:2.3.5.1-1   
>> dovecot-2.3-latest4.4 M dovecot-imaptestx86_64 
>>  2:2.3.5.1-1dovecot-2.3-latest 79 k
>> dovecot-lua x86_642:2.3.5.1-1   
>> dovecot-2.3-latest100 k Installing for dependencies:
>> clucene-corex86_642.3.3.4-11.el7
>> base  528 k
>> 
>> Transaction Summary
>> 
>> == Install  3 Packages (+1 Dependent
>> package)
>> 
>> ..
>> 
>> Running transaction
>>  Installing : clucene-core-2.3.3.4-11.el7.x86_64   
>> 1/4 Installing : 2:dovecot-2.3.5.1-1.x86_64   
>> 2/4 Installing
>> : 2:dovecot-imaptest-2.3.5.1-1.x86_64  
>> 3/4 Installing : 2:dovecot-lua-2.3.5.1-1.x86_64   
>> 4/4 Verifying  :
>> clucene-core-2.3.3.4-11.el7.x86_64 
>>   1/4 Verifying  : 2:dovecot-imaptest-2.3.5.1-1.x86_64
>>   2/4 Verifying  :
>> 2:dovecot-2.3.5.1-1.x86_64 
>>   3/4 Verifying  : 2:dovecot-lua-2.3.5.1-1.x86_64 
>>   4/4
>> 
>> Installed:
>>  dovecot.x86_64 2:2.3.5.1-1  dovecot-imaptest.x86_64 2:2.3.5.1-1 
>> dovecot-lua.x86_64 2:2.3.5.1-1
>> 
>> Dependency Installed:
>>  clucene-core.x86_64 0:2.3.3.4-11.el7
>> 
>> -- K
> 
> 
> -- 
> mit freundliche Grüßen / best regards,
> 
>  Günther J. Niederwimmer
> 
> 



Re: v2.2.27 Panic: file rfc822-parser.h: line 23 (rfc822_parser_deinit): assertion failed: (ctx->data <= ctx->end)

2019-03-28 Thread Sami Ketola via dovecot


> On 28 Mar 2019, at 1.08, Jason Lewis via dovecot  wrote:
> 
> Thanks Timo.
> 
> Given the age of these dovecot packages, and this being on debian
> oldstable, what should we do next? I'm inclined to just delete the email
> in question and move on.
> 

https://repo.dovecot.org/ 

Sami

Re: Maildir permissions issue with Postfix

2019-03-28 Thread Sami Ketola via dovecot


> On 27 Mar 2019, at 17.03, Randall R. Sargent via dovecot 
>  wrote:
> 
> Hi all,
>  
> We have a Postfix server that serves as an alias MTA to route to other mail 
> systems. I’ve recently installed Dovecot on it because we have three service 
> accounts that need simple POP3 mailboxes. I have the accounts set up on the 
> system and mail does get delivered to their ~/Maildir/ locations by Postfix, 
> however, every time a message is dropped in the Maildir for the user, the 
> permissions for the message are set to the user’s uid:gid and Dovecot can’t 
> access the mail message. Doing a chmod fixes it temporarily until the next 
> message drops.  The vmail group has access to the user’s home folder 
> structure.
>  
> Any help is appreciated!!

use dovecot-lda to deliver the emails. https://wiki.dovecot.org/LDA/Postfix

Sami



Re: quota-service with Director - A workaround

2019-03-23 Thread Sami Ketola via dovecot


> On 23 Mar 2019, at 14.28, Tom Sommer via dovecot  wrote:
> 
> On 2019-03-21 10:28, Sami Ketola via dovecot wrote:
>>> On 20 Mar 2019, at 18.17, Tom Sommer via dovecot  
>>> wrote:
>>> On 2019-03-20 16:40, Sami Ketola via dovecot wrote:
>>>>> On 20 Mar 2019, at 17.13, Tom Sommer via dovecot  
>>>>> wrote:
>>>>> I realize quota-service on Director is not supported, which is a shame.
>>>>> As a workaround I'm thinking of setting up quota-service on one of my 
>>>>> backend nodes, and have all my Postfix services ask this one node for the 
>>>>> quota status.
>>>>> This sort of defeats the purpose of the Director (having per-user 
>>>>> assigned hot nodes), since now this one node running the quota-service 
>>>>> will access all mailboxes to check the status of all inbound mail.
>>>>> Is this a problem though? In terms of NFS locking etc. etc.?
>>>> Might be. Wouldn't it be just easier to use the overquota-flag available 
>>>> since 2.2.16 and set up overquota flag in LDAP or userdb of choice and 
>>>> configure postfix to check that flag?
>>> I don't really want to involve LDAP in my setup :)
>> So use what ever your shared userdb service is as you must have one if
>> you are using multiple backends and directors.
> 
> Does it work with mysql userdb? Is there an example to look at anywhere?


Quota over flag is explained in wiki 
https://wiki2.dovecot.org/Quota/Configuration 
<https://wiki2.dovecot.org/Quota/Configuration>

Basically you just need to return quota_over_flag from userdb and make a script 
that updates the flag in mysql in case the flag is incorrect.
And then also configure postfix to check the same flag.

Sami



Re: quota-service with Director - A workaround

2019-03-21 Thread Sami Ketola via dovecot



> On 20 Mar 2019, at 18.17, Tom Sommer via dovecot  wrote:
> 
> 
> On 2019-03-20 16:40, Sami Ketola via dovecot wrote:
> 
>>> On 20 Mar 2019, at 17.13, Tom Sommer via dovecot  
>>> wrote:
>>> I realize quota-service on Director is not supported, which is a shame.
>>> As a workaround I'm thinking of setting up quota-service on one of my 
>>> backend nodes, and have all my Postfix services ask this one node for the 
>>> quota status.
>>> This sort of defeats the purpose of the Director (having per-user assigned 
>>> hot nodes), since now this one node running the quota-service will access 
>>> all mailboxes to check the status of all inbound mail.
>>> Is this a problem though? In terms of NFS locking etc. etc.?
>> Might be. Wouldn't it be just easier to use the overquota-flag available 
>> since 2.2.16 and set up overquota flag in LDAP or userdb of choice and 
>> configure postfix to check that flag?
> 
> I don't really want to involve LDAP in my setup :)

So use what ever your shared userdb service is as you must have one if you are 
using multiple backends and directors.

Sami



Re: imap ---- LIST "" * The returned mailbox does not display quotes

2019-03-21 Thread Sami Ketola via dovecot


> On 21 Mar 2019, at 10.08, lty via dovecot  wrote:
> 
>  
> Yes,
> Foxmail for mac client, after dovecot upgrade from 2.1.17 to new version, 
> imap can't receive any email, I perform packet capture test, foxmail will not 
> have next step after sending {LIST "" *} command action.
> 
> Foxmail for mac client, dovecot version 2.1.17 can receive imap mail 
> normally, because we need to add quota_max_mail_size configuration, we must 
> upgrade to a higher version (https://wiki2.dovecot.org/Quota/Configuration# 
> Maximum_saved_mail_size)
> 
> 
> After comparison, the new version of dovecot returned without quotes.
> I hope that the quotes can be changed in the configuration, can I?
> 

No.

> Can I change the configuration to add quotes now?
>   Or add this setting in the next version.
> 


No.

It's still Foxmail that is broken here. Please contact the authors of Foxmail 
so that they can make their software RFC compliant.
Or stop using Foxmail.

Sami



Re: quota-service with Director - A workaround

2019-03-20 Thread Sami Ketola via dovecot


> On 20 Mar 2019, at 17.13, Tom Sommer via dovecot  wrote:
> 
> I realize quota-service on Director is not supported, which is a shame.
> 
> As a workaround I'm thinking of setting up quota-service on one of my backend 
> nodes, and have all my Postfix services ask this one node for the quota 
> status.
> 
> This sort of defeats the purpose of the Director (having per-user assigned 
> hot nodes), since now this one node running the quota-service will access all 
> mailboxes to check the status of all inbound mail.
> 
> Is this a problem though? In terms of NFS locking etc. etc.?


Might be. Wouldn't it be just easier to use the overquota-flag available since 
2.2.16 and set up overquota flag in LDAP or userdb of choice and configure 
postfix to check that flag?

https://wiki2.dovecot.org/Quota/Configuration 


Sami



Re: Upgrade from 2.1.17 2.2.36

2019-03-19 Thread Sami Ketola via dovecot


> On 19 Mar 2019, at 10.26, lty via dovecot  wrote:
> 
> Thank you for your reply
> 
>  
> But I tested it in dovecot v2.1.17, imap can receive mail normally.
> I want to ask if the returned data can be removed after the [ LIST "" * ] 
> command?
> Or can you remove some parameters?
> 
> I also observed that v2.1.17 and v2.2.36 are different.
> After the [ LIST "" * ] command,
> The v2.1.17 directory has quotes.
> The v2.2.36 directory has no quotes.
> 
> [* LIST (\HasNoChildren) "/" "INBOX" ] where ["INBOX" ] Can I add quotes or 
> remove quotes from the configuration file?
> 
> 

That is a negative for both. And also both are RFC compliant. Software 
application that can't parse those is broken.

Sami




Re: Upgrade from 2.1.17 2.2.36

2019-03-19 Thread Sami Ketola via dovecot


> On 19 Mar 2019, at 3.31, lty via dovecot  wrote:
> 
> I use the MAC Foxmail client, v2.2.36 can't receive mail, but v2.1.17 can 
> receive mail.
> I grabbed the network package and found that Foxmail was not processing the 
> next step after sending the D LIST "" * command.
> I suspect that because of the time of return, Foxmail is not recognized, so I 
> want to remove this time.
> 
> 

Replicated the issue. This seems to be a bug in Foxmail. I tested version 
1.2.0.93608. Please contact the authors of Foxmail to get this bug fixed.

Sami



Re: quota_max_mail_size no tips

2019-03-05 Thread Sami Ketola via dovecot
Do you have the quota plugin loaded?

Sami

> On 4 Mar 2019, at 11.46, 默认 via dovecot  wrote:
> 
>  
> When the link dovecot imports the message, after the rejection, there is no 
> prompt in the log.
> 
>  I want to get the prompt log of the import failure, what should I do?
> 
>>  
>> [root@test5 dovecot]# dovecot --version 
>> 2.2.36 (1f10bfa63)
>> 
> plugin {
>   quota_max_mail_size = 100M
> }
>  



Re: index problems after update

2019-02-21 Thread Sami Ketola via dovecot



> On 21 Feb 2019, at 12.23, Hajo Locke via dovecot  wrote:
> I think mbox+procmail is a classic setup and wide used and good solution for 
> many usecases. Same setup we use many years.
> We run ~2 mio mailboxes. our automated systems depends on this setup. 
> creating mailboxes, managing mailboxes, creating automated filterrules, 
> backupsystem to tell something of them. we can not switch our whole mailsetup 
> to work around this bug.
> How to get a dump if dovecot not crashing but has wrong behaviour? I would 
> like to help and provide useful info, but it depends on kind of problem.
> I think if a classic setup is not working in dovecot any more, this is a 
> serious problem.

In you first email to this thread it says:

> Feb  8 08:45:37 hostname dovecot[14882]: imap(myuser): Fatal: master: 
> service(imap): child 14135 killed with signal 6 (core dumped)

So imap is crashing and even dumping a core.

Also I must disagree with your mbox+procmail statement. mbox has always been 
very unoptimised mailbox format and everyone should be emphasised not to use it.
Also that combination has always had problems with indexing and file locking. I 
would not use it on high volume mailservers. Or even medium volume mailservers.

Sami



Re: Assistance with doveadm backup...

2019-02-20 Thread Sami Ketola via dovecot



> On 21 Feb 2019, at 7.04, SH Development via dovecot  
> wrote:
> 
> Not really.  The suggestions in the post seemed to be:
> 
> 1. Tar the whole directory, which I have tried in the past, but a 130G 
> directory takes over 18 hours to tar.
> 2. It was also suggested to rsync the directory, but the question was brought 
> up, and not answered, about whether it was advisable to copy live mail, thus 
> the need for doveadm sync/backup.
> 
> Based on what I’ve read about doveadm backup, this is what I need, I am just 
> having a hard time formulating the syntax for the command.


root@ketola ~]# mkdir -p /backup/sami
root@ketola ~]# chown vmail:vmail /backup/sami
root@ketola ~]# doveadm backup -u sami Maildir:/backup/sami

root@ketola ~]# du -hs /backup/sami
211M /backup/sami

does this help?

Sami



Re: Error sending email from client

2019-02-18 Thread Sami Ketola via dovecot



> On 18 Feb 2019, at 15.29, Latin Bishop via dovecot  
> wrote:
> 
> Here is error when sending
> Feb 18 08:24:27 pepino dovecot: imap-login: Login: user=, method=PLAIN, 
> rip=127.0.0.1, lip=127.0.1.1, mpid=5297, TLS, session=<1gQ/CyuChpB/AAAB>
> Feb 18 08:24:28 pepino dovecot: imap(pi): Logged out in=8 out=416
> Feb 18 08:24:34 pepino dovecot: imap-login: Login: user=, method=PLAIN, 
> rip=127.0.0.1, lip=127.0.1.1, mpid=5303, TLS, session=
> Feb 18 08:24:59 pepino postfix/smtpd[5278]: connect from localhost[127.0.0.1]
> Feb 18 08:24:59 pepino postfix/smtpd[5278]: warning: TLS library problem: 
> error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown 
> ca:../ssl/re$Feb 18 08:24:59 pepino postfix/smtpd[5278]: lost connection 
> after STARTTLS from localhost[127.0.0.1]
> Feb 18 08:24:59 pepino postfix/smtpd[5278]: disconnect from 
> localhost[127.0.0.1] ehlo=1 starttls=1 commands=2
> Feb 18 08:26:41 pepino postfix/smtps/smtpd[5328]: connect from 
> localhost[127.0.0.1]
> Feb 18 08:26:41 pepino postfix/smtps/smtpd[5328]: warning: unknown smtpd 
> restriction: "permit_ssl_authenticated"
> Feb 18 08:26:41 pepino postfix/smtps/smtpd[5328]: NOQUEUE: reject: RCPT from 
> localhost[127.0.0.1]: 451 4.3.5 Server configuration error; from= 18 08:26:44 pepino postfix/cleanup[5336]: 8B85C80DE9: 
> message-id=<20190218132644.8b85c80...@mail.torombolo.ml>
> Feb 18 08:26:44 pepino postfix/smtps/smtpd[5328]: disconnect from 
> localhost[127.0.0.1] ehlo=1 mail=1 rcpt=0/1 quit=1 commands=3/4
> Feb 18 08:26:44 pepino postfix/qmgr[4714]: 8B85C80DE9: 
> from=, size=916, nrcpt=1 (queue active)
> Feb 18 08:26:44 pepino postfix/local[5337]: 8B85C80DE9: 
> to=, orig_to=, relay=local, delay=0.1, 
> delays=0.05/0.03/0/0.02,$Feb 18 08:26:44 pepino postfix/qmgr[4714]: 
> 8B85C80DE9: removed
> 

This is postfix configuration error and not a Dovecot issue.

Sami

Re: Error configuring unix_listener stats-writer

2019-02-18 Thread Sami Ketola via dovecot



> On 18 Feb 2019, at 13.12, Peter Nabbefeld via dovecot  
> wrote:
> 
> 
> 
> Am 18.02.19 um 11:34 schrieb Sami Ketola:
>> 
>>> On 18 Feb 2019, at 12.15, Peter Nabbefeld  wrote:
>>> 
>>> 
>>> 
>>> Am 17.02.19 um 18:27 schrieb Sami Ketola:
>>>> Easy way is to use just one config file. That is what I have done for 
>>>> years.
>>>> 
>>>> Sami
>>>> 
>>>> 
>>> I'd preferred that, too, but it seems the 'modern' way - and I don't want 
>>> to get any possible configuration issues on updates, so I won't change this.
>> 
>> If your distribution modifies config files on updates then it's time to 
>> switch to another distribution.
>> 
>> Sami
> Not the config files itself. But IIRC, the "main" config files have been 
> changed in distros when the new style was introduced. Usually the main file 
> wasn't overwritten, but a new file with an extension like "new" had been 
> added.

Shame on those distros.

However even if you do use such lousy distro, you can work around it by just 
having ONE config file in the conf.d folder. That would then be the real config 
file.

> 
> However, my laptop installation isn't as old, and every application I install 
> already has the new style, and of course it has it pros when You get used to 
> the structure of the config file, but if You're new to some application, You 
> also have to learn about this structure for every new one ...

I wish that the split config default would have never been introduced. It's a 
nightmare.

Sami



Re: Error configuring unix_listener stats-writer

2019-02-18 Thread Sami Ketola via dovecot



> On 18 Feb 2019, at 12.15, Peter Nabbefeld  wrote:
> 
> 
> 
> Am 17.02.19 um 18:27 schrieb Sami Ketola:
>>> 
>> 
>> Easy way is to use just one config file. That is what I have done for years.
>> 
>> Sami
>> 
>> 
> I'd preferred that, too, but it seems the 'modern' way - and I don't want to 
> get any possible configuration issues on updates, so I won't change this.


If your distribution modifies config files on updates then it's time to switch 
to another distribution.

Sami

Re: Error configuring unix_listener stats-writer

2019-02-18 Thread Sami Ketola via dovecot
On 17 Feb 2019, at 11.47, Peter Nabbefeld via dovecot  wrote:Is there any tool displaying an "effective" config tree (i.e., a tool displaying the resulting configuration from every file, displaying the original file and line number)? "dovecot -a" only displays all configurations, but without source reference, so it's impossible to find out about the details.  :-(Easy way is to use just one config file. That is what I have done for years.Sami


Re: Archive maildir

2019-02-13 Thread Sami Ketola via dovecot


> On 13 Feb 2019, at 11.22, Gandalf Corvotempesta via dovecot 
>  wrote:
> 
> Hi to all
> We have a maildir with about 180GB of emails.
> We have to archive them to a structure like: .Archive.YYY./MM.folder
> 
> Are you aware of a script doing this ? I've found a perl script that
> doesn't spit in year and month and a very, very, very, very, very old
> python script that:
> 1) doesn't manage base64 encoded subject properly
> 2) doesn't work with python 3.x (that is able to manage base64 encoded
> subject properly
> 
> Any idea ?

Maybe just simple scripting and doveadm move? There is even example at the 
bottom of the wikipage:
https://wiki2.dovecot.org/Tools/Doveadm/Move 


Sami

Re: Dovecot v2.2.36.1 released

2019-02-05 Thread Sami Ketola via dovecot
Hi,

It's probably because gmail. They refuse emails for random reasons occasionally.

Sami

> On 5 Feb 2019, at 17.06, Larry Rosenman  wrote:
> 
> for some reason Aki's posts are not making it to my GMail account from this 
> list.
> 
> Any idea why?
> 
> On Tue, Feb 5, 2019 at 10:04 AM Eric Broch  > wrote:
> Thank you!
> 
> On 2/5/2019 8:43 AM, Aki Tuomi wrote:
>> Hi,
>> 
>> as per our EOL statement 2.2.36 receives security and critical updates. That 
>> said, we decided to flush few annoying bugs with .1 release.
>> 
>> You do not need to build releases for 2.2.
>> 
>> Aki
>>> On 05 February 2019 at 17:36 Eric Broch < ebr...@whitehorsetc.com 
>>> > wrote:
>>> 
>>> 
>>> Aki,
>>> 
>>> What's the difference between 2.2.x and 2.3.x version of Dovecot? And
>>> why do you maintain both?
>>> 
>>> I stopped building RPM's of the 2.2.x version and now only build 2.3.x.
>>> Should I be maintaining both?
>>> 
>>> Eric
>>> 
>>> On 2/5/2019 6:01 AM, Aki Tuomi wrote:
 https://dovecot.org/releases/2.2/dovecot-2.2.36.1.tar.gz 
 
 https://dovecot.org/releases/2.2/dovecot-2.2.36.1.tar.gz.sig 
 * 
 CVE-2019-3814: If imap/pop3/managesieve/submission client has
   trusted certificate with missing username field
   (ssl_cert_username_field), under some configurations Dovecot
   mistakenly trusts the username provided via authentication instead
   of failing.
 * ssl_cert_username_field setting was ignored with external SMTP AUTH,
   because none of the MTAs (Postfix, Exim) currently send the
   cert_username field. This may have allowed users with trusted
   certificate to specify any username in the authentication. This bug
   didn't affect Dovecot's Submission service.
 - pop3_no_flag_updates=no: Don't expunge RETRed messages without QUIT
 - director: Kicking a user assert-crashes if login process is very slow
 - lda/lmtp: Fix assert-crash with some Sieve scripts when
   mail_attachment_detection_options=add-flags-on-save
 - fs-compress: Using maybe-gz assert-crashed when reading 0 sized file
 - Snippet generation crashed with invalid Content-Type:multipart
>>> >
 ---
 Aki Tuomi
 Open-Xchange Oy
>>> >
>>> --
>>> Eric Broch
>>> White Horse Technical Consulting (WHTC)
>> 
>> --- 
>> Aki Tuomi
> -- 
> Eric Broch
> White Horse Technical Consulting (WHTC)
> 
> 
> -- 
> Larry Rosenman http://www.lerctr.org/~ler 
> 
> Phone: +1 214-642-9640 (c) E-Mail: larry...@gmail.com 
> 
> US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106



Re: problem in setting up proxy

2019-01-23 Thread Sami Ketola



> On 23 Jan 2019, at 16.33, Ted  wrote:
> 
> Hello,
> 
> Absolutely, thanks, here it is:
> 
> passdb {
>   args = nopassword=y
>   default_fields = proxy=y host=10.5.10.121
>   driver = static
>   name = static
> }

at least change this to:

passdb {
  driver = static
  name = static
  args = nopassword=y  proxy=y host=10.5.10.121
}

but not entirely sure if there is something else.

Sami


Re: Moving messages between servers with different configurations

2019-01-22 Thread Sami Ketola
Hi,

not easily. You can always limit dsync to a single folder with -m folder, but 
then again it's designed to make 1:1 copy so you would need to rename the 
synced folder afterwards.

Sami

> On 22 Jan 2019, at 17.55, Marc Roos  wrote:
> 
> 
> This pulls over the whole test mailbox, but I need it to be stored in a 
> different mailbox, of course that is now possible to move from the local 
> test to the local INBOX/test2. But then I might aswell scp the mbox in 
> the first place. 
> 
> Should we conclude that it is not possible with a single doveadm command 
> to move messages from RemoteServerA/usertest/mailboxA/messagesof2017 to 
> LocalServerB/usertest/mailboxX/ ?
> 
> [@~]# doveadm mailbox list -u testuser
> Drafts
> Junk
> Trash
> INBOX/test2
> INBOX/test3
> Sent
> Archive
> Archive/Archive
> Archive/2019old
> INBOX
> [@~]# doveadm backup -u testuser -R -m test tcp:192.168.10.43:542
> [@~]# doveadm mailbox list -u testuser
> Drafts
> Junk
> Trash
> INBOX/test2
> INBOX/test3
> Sent
> test   <--- new!
> Archive
> Archive/Archive
> Archive/2019old
> INBOX
> 
>> try running this on the NEW dovecot server:
>> 
>> doveadm backup -u uid -R tcp:192.168.10.43:542
>> 
>> to pull the messages from old server.
>> 
>>> I feel a bit like an idiot, but I have been trying with copy, so I 
> do 
>>> not lose any message when testing. But I can't get them to copy. I 
> do 
>>> indeed have same uid and gid.
>>> 
>>> 
>>> [@~]# doveadm mailbox status -S 192.168.10.43:542 -u testuser -t 
>>> messages test
>>> messages=43
>>> [@~]# doveadm mailbox status -u testuser -t messages INBOX/test2
>>> messages=16
>>> 
>>> [@~]# doveadm -v copy -S 192.168.10.43:542 -u testuser INBOX/test2 
>>> mailbox test
>>> [@~]#
>>> 
>>> [@~]# doveadm mailbox status -S 192.168.10.43:542 -u testuser -t 
>>> messages test
>>> messages=43
>>> [@~]# doveadm mailbox status -u testuser -t messages INBOX/test2
>>> messages=16 
>>> 
>>> Also tried with
>>> doveadm -v copy -S 192.168.10.43:542 -u testuser INBOX/test2 user 
>>> testuser mailbox test
>>> 
>>> 
>>> 
 doveadm [-Dv] move [-S socket_path] -u user  destination [user 
>>> source_user] search_query
 
 Moving all mails from mailbox INBOX/test on serverA to mailbox
 Archive/2017 on local serverB.
 + destination Archive/2017 must exist
 + Limitation: source_user and testuser must share the same UID and 
>>> GID
 
 doveadm move -S x.x.x.x:x -u testuserAtServerB
 Archive/2017 user source_userAtServerA mailbox INBOX/test ALL
 
 
> I wanted to move messages from a mbox mailbox on server A to mdbox
> on 
> server B. I thought I could do this by connecting to the remote
> server 
> with "doveadm move -S x.x.x.x:x -u testuser Archive/2017 mailbox 
> INBOX/test" but I guess this will only allow and move messages 
> internally on server B?
> Should I use dsync, or is there another way to move the messages?
> 



Re: Moving messages between servers with different configurations

2019-01-22 Thread Sami Ketola
Hi,

try running this on the NEW dovecot server:

doveadm backup -u uid -R tcp:192.168.10.43:542

to pull the messages from old server.

Sami


> On 22 Jan 2019, at 16.52, Marc Roos  wrote:
> 
> 
> 
> Hi Martin,
> 
> I feel a bit like an idiot, but I have been trying with copy, so I do 
> not lose any message when testing. But I can't get them to copy. I do 
> indeed have same uid and gid.
> 
> 
> [@~]# doveadm mailbox status -S 192.168.10.43:542 -u testuser -t 
> messages test
> messages=43
> [@~]# doveadm mailbox status -u testuser -t messages INBOX/test2
> messages=16
> 
> [@~]# doveadm -v copy -S 192.168.10.43:542 -u testuser INBOX/test2 
> mailbox test
> [@~]#
> 
> [@~]# doveadm mailbox status -S 192.168.10.43:542 -u testuser -t 
> messages test
> messages=43
> [@~]# doveadm mailbox status -u testuser -t messages INBOX/test2
> messages=16 
> 
> Also tried with
> doveadm -v copy -S 192.168.10.43:542 -u testuser INBOX/test2 user 
> testuser mailbox test
> 
> 
> 
>> doveadm [-Dv] move [-S socket_path] -u user  destination [user 
> source_user] search_query
>> 
>> Moving all mails from mailbox INBOX/test on serverA to mailbox
>> Archive/2017 on local serverB.
>> + destination Archive/2017 must exist
>> + Limitation: source_user and testuser must share the same UID and 
> GID
>> 
>> doveadm move -S x.x.x.x:x -u testuserAtServerB
>> Archive/2017 user source_userAtServerA mailbox INBOX/test ALL
>> 
>> 
>>> I wanted to move messages from a mbox mailbox on server A to mdbox
>>> on 
>>> server B. I thought I could do this by connecting to the remote
>>> server 
>>> with "doveadm move -S x.x.x.x:x -u testuser Archive/2017 mailbox 
>>> INBOX/test" but I guess this will only allow and move messages 
>>> internally on server B?
>>> Should I use dsync, or is there another way to move the messages?
>>> 
>>> 
>>> 
> 



Re: How to connect to a remote server and execute eg a search

2019-01-21 Thread Sami Ketola


> On 21 Jan 2019, at 22.00, Marc Roos  wrote:
> 
> 
> 
> Thanks Sami, And I should allow access on the remote server via 
> login_access_sockets? And the authenticated user via this connection is 
> then testuser I presume?
> 


No. It's the doveadm service that it's contacting:

service doveadm {
  service_count = 1000
  inet_listener {
   port = 2425
  }
}

and passwords need to match on both servers:

doveadm_password = secret

Sami

Re: How to connect to a remote server and execute eg a search

2019-01-21 Thread Sami Ketola



> On 21 Jan 2019, at 21.32, Marc Roos  wrote:
> 
> 
> Is there a page that explains how to connect to a remote server? What is 
> the syntax of socket_path?
> 
> 
> doveadm search -S XXX -u testuser mailbox INBOX ALL

-S :

Sami


Re: Authentication lost within session

2019-01-17 Thread Sami Ketola



> On 17 Jan 2019, at 14.19, Marc Roos  wrote:
> 
> 
> Same ip, connection, session happens after a few seconds, all on local 
> test network.

Sorry I mistakenly overlooked at the ip:s. But anyway that is multiple 
connections and seems that PAM has some concurrency limit and refuses the third 
connection Thunderbird opens.

so you probably have pam_limits with maxlogins=2 enabled.

Sami



Re: Authentication lost within session

2019-01-17 Thread Sami Ketola



> On 17 Jan 2019, at 14.09, Marc Roos  wrote:
> 
> When I open the mail client (thunderbird), I can access the mailboxes 
> and all of a sudden, I am loosing the authenticated session. Any idea's 
> where to look?
> 
> 
> 
> 
> Jan 17 12:42:04 mail04 dovecot: imap-login: Login: user=, 
> method=PLAIN, rip=192.168.10.219, lip=192.168.10.44, mpid=13403, TLS, 
> session=



> Jan 17 12:42:14 mail04 dovecot: auth-worker(13400): 
> pam(usertest,192.168.10.219,): pam_authenticate() 
> failed: Authentication failure (password mismatch?)


So is this another connection or does the ip-address change mid session?

Sami



Re: Accidentally deleted .imap folder, how can I reindex?

2019-01-17 Thread Sami Ketola
Hi,

doveadm force-resync -u user FOLDER

Sami

> On 17 Jan 2019, at 13.32, Les  wrote:
> 
> 
> Hello,
> 
> I have several users with Maildir format. After upgrading from 2.2 to 2.3, 
> some directories could not be opened from thunderbird. The server log shows 
> all kinds of errors including these:
> 
> Corrupted record in index cache file 
> /home/user01/Maildir/.Elk/dovecot.index.cache: UID 2802: Broken 
> physical size in mailbox 
> Cached message size larger than expected (18 > 17, box=Sent, UID=2802)
> 
> While trying to fix the problem, I have accidentally deleted some of the 
> index/cache directories. In particular, Maildir/.Sent/.imap and  
> Maildir/.Sent/cur/.imap folders were deleted, but all of the mail message 
> files are there. When I try to open the "Sent" folder from thunderbird, it 
> does not show all of the messages. It only shows messages until 2019.01.12. 
> If I send out an email from thunderbird, then a new message is copied into 
> the Maildir/.Sent/ folder, but it is not visible from thunderbird. I have 
> already tried to unsubscribe + subscribe to the Sent folder, but it did not 
> work - it re-downloads all messages until 2019.01.12, but not after that.
> 
> Can I somehow regenerate the index files, force dovecot to recreated .imap 
> foder, and force thunderbird to re-download and show all of the files somehow?
> 
> Thank you,
> 
>Laszlo
> 



Re: Import mailbox from different domain

2019-01-14 Thread Sami Ketola


> On 14 Jan 2019, at 22.47, Sergio Belkin  wrote:
> 
> Hi folks,
> Let's say that I have on dovecot the domain example.net  
> and on MS Exchange example.com .
> I've tried to import a mailbox from MS Exchange to Dovecot but it fails, I've 
> run:
> 
>  dsync -Dv   backup -R  -u joe.doe  imapc:
> 
> dsync(joe@example.net ): Error: 
> imapc(192.168.0.2:993 ): Authentication failed: 
> AUTHENTICATE failed.
> 
> Is there a way to submit different domains both on MS Exchange and Dovecot 
> and to make everyone be happy :)  ?

like -o imapc_user=joe@example.com  
-o imapc_password=joe.does.password?

Sami



Re: mdbox import error from read-only filesystem

2019-01-14 Thread Sami Ketola
Hi,

you can use INDEX=/writable/path/%u in your mail_location setting to define 
location for the required index data when importing.
Also possibly you would need to define writable location for CONTROL and 
VOLATILEDIR. 

see https://wiki.dovecot.org/MailLocation 


Sami

> On 14 Jan 2019, at 11.23, hby  wrote:
> 
> Dovecot version: 2.2.34
> 
> doveadm import tries to call open() on the source indexes/logs of mdbox data, 
> but even if it should work as it is just a read-related call, it fails on 
> read-only filesystems.
> 
> The main use case for read-only filesystem is restoring from backup: the 
> massive deduplicated backup data is exposed using FUSE. For deduplication to 
> work efficiently, we have to backup the whole mail storage and thus the only 
> way to handle the restore to mount the whole data.
> 
> I would like to ask for some proper solution for this problem.
> 
> Suggestions: first, the doveadm import should never modify the indexes, and 
> second: the open() calls should respect the read-only filesystem and should 
> fall back to "r" flag when using open.
> The problem goes further when using SIS (which goes beyond multiple 
> terrabytes).
> 
> The only workaround currently is to export the whole mdbox data to a 
> writeable storage, and parametrize the import command to use that and the 
> mounted SIS data separately for import...which is just problematic, ugly and 
> error prune.
> 
> Thank you for the continous work on dovecot, I hope you get this feedback in 
> a good way.



Re: doveadm + HA

2019-01-07 Thread Sami Ketola


> On 7 Jan 2019, at 16.02, Maciej Milaszewski IQ PL  
> wrote:
> 
> Hi
> I have two server directors in ring and 5 dovecot servers (2.2.36)
> IP for IMAP and POP3 is a VIP (keepalived)
> 
> 
> What is the best solutions to get realy HA for 5 dovecot servers ?
> Maby corosync+pacemeker ? But this solution is too problematic and hardcore
> 
> Why I need HA ?
> Doveadmin is too lazy and doveadm director does not know that one
> machine broke down and still sends traffic
> 


Dovecot is HA out of the box with shared storage and directors. Also you should 
check this for backend monitoring:

https://github.com/brandond/poolmon 

Sam

Re: Compiling Dovecot on Solaris 11 fails

2019-01-03 Thread Sami Ketola


> On 3 Jan 2019, at 18.45, Andrew Watkins  wrote:
> 
> 
> Hi,
> 
> Tried to build dovecot-2.3.4 on Solaris 11 x86 and it fails at configure 
> part. I just went checked and it last version it works on is v2.3.2.1


does work just fine on my solaris 11 x86 box.

configure:22610: checking whether fd passing works
configure:22685: gcc -o conftest -std=gnu99 -g -O2 -fstack-protector 
-U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -Wall -W -Wmissing-prototypes 
-Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 
-Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -I./src/lib 
./src/lib/fdpass.c   conftest.c -lsocket -lnsl -lresolv  >&5
configure:22685: $? = 0
configure:22685: ./conftest
configure:22685: $? = 0
configure:22708: result: yes

What is your Solaris release and which version of gcc you are using?

my test system is solaris 11.3 and I have gcc 4.5.2. 

Sami




Re: gcc -> clang

2019-01-03 Thread Sami Ketola



> On 3 Jan 2019, at 14.04, Ruben Safir  wrote:
> 
> On 1/3/19 4:50 AM, Rupert Gallagher via dovecot wrote:
>> Please, use clang instead of gcc. Code quality can only profit from it. I 
>> just compiled 2.3.4 and compiler stderr is full of interesting problems.
>> 
> 
> 
> oh please


This is not very helpful.

Sami

Re: Several problems on Solaris10

2019-01-01 Thread Sami Ketola


> On 31 Dec 2018, at 19.28, Pierluigi Frullani  
> wrote:
> 
> 
> Answer inline.
> 
> On Sun, Dec 30, 2018 at 12:59 PM James  > wrote:
> On 29/12/2018 13:49, Pierluigi Frullani wrote:
> 
> > My version is 2.2.13 ( it was the last one, at the time of the first
> > server setup ).
> 
> 2.2.13 is from around May 2014.  It worked but I can't see why you 
> wouldn't switch to the latest 2.3.4. (You might be seeing what I can't 
> and your question hasn't explained.)
> 
> That's the date for installation. I was using the courier-imap and switched 
> to dovecot. 
> Not changed since then. 
>  
> > I have seen that ( it seems ) the new solaris don't honour the
> > LD_LIBRARY_PATH.
> 
> I'm sure it does but you shouldn't need it anyway.
> 
> Believe me, it doesn't :( 

Believe me it does. I used to work for Sun Microsystems for 14 years in Solaris 
support and sustaining and I can guarantee you that it does.

You problem is that Solaris has concept of Secure Runtime Linker, and for 
trusted applications most of LD_CONFIG and LD_LIBRARY_PATH is ignored for 
security reasons.

For secure applications LD_LIBRARY_PATH components are ignored for non-secure 
directories.

Your dovecot is probably setuid or setgid and considered as secure application 
and secure runtime linker rules are triggered for it. Then /usr/local is 
completely ignored from LD_LIBRARY_PATH.

Sami



Re: Migration

2018-12-31 Thread Sami Ketola



> On 31 Dec 2018, at 12.22, Ignacio García  wrote:
> 
> A totally different approach (that is imap-server agnostic), providing that 
> you're setting up those new accounts with temporary passwords (which you 
> know), before users change their passwords to their liking: you could also 
> use imapsync ( https://github.com/imapsync/imapsync) . We here use it with a 
> batch file and a text file containing all accounts to do mass-migrations, 
> usually at night, when there's little to none user interaction with their 
> mail accounts. I like this approach because mail service never gets 
> interrupted and we do programmed syncs all night in case DNS propagation 
> takes more than expected and mail still arrives to the old server.


I still don't recommend to use imapsync for migrations as UIDVALIDITY and UID 
numbers will change and end users will need to invalidate their local client 
caches and redownload all mail headers and usually also mail bodies again.

Sami

Re: Issue with LMTP proxying and port number

2018-12-30 Thread Sami Ketola

> On 31 Dec 2018, at 0.43, Laz C. Peterson  wrote:
> 
> Hello there,
> 
> Everything was working fine on Dovecot 2.2.10 (on CentOS 7), but after 
> updating to version 2.2.36, our director servers are not able to proxy LMTP.
> 
> We are sending mail from exim to the local Dovecot LMTP socket, which then 
> used to send it to our internal mail servers on port 24.
> 
> Besides the update, nothing has changed in the config files.  The 
> configuration has been working great for months … Then we had a great idea of 
> doing holiday maintenance.
> 
> (Currently, we are working around this issue by having exim send mail to the 
> IP and port, instead of the socket.)
> 
> Doing a tcpdump capture shows no TCP traffic either.  The logs seem to say 
> that it is trying to go to port 0, so my guess is that it’s not getting the 
> port directive from either static or sql — since the docs state that if these 
> settings aren't specified, it will always use the same connection type for 
> the proxy that it received the connection on.
> 
> Any thoughts?  I can share config if necessary.
> 
> ~ Laz Peterson
> Paravis, LLC
> 


can you please post doveconf -n and your sql userdb setup?

Sami

Re: Migration

2018-12-30 Thread Sami Ketola


> On 29 Dec 2018, at 23.49, Hans Brage  wrote:
> 
> Hi!
> 
> I'm currently running an small imap-server on Dovecot 2.2.4 but will retire 
> that server. I've set up an new server with Dovecot 2.3.4 and will migrate 
> the mailboxes (maildir-format) from the old to the new server. I'm planning 
> to restructure the accounts a bit when migrating so I need to move them one 
> by one. Its only a few so thats not a big issue.
> 
> Both servers are using virtual accounts (users file).
> 
> I've understood that best method for migration is to run doveadm backup -R 
> from the new server. But I really cant figure out what other parameters or 
> settings that I need to perform that task and if it should be run as user 
> vmail or root.
> 
> Would be greatful for any tips, hints, links or similiar.


Once you have the new server set up, you can use dsync over ssh to sync the 
mailboxes with something like:

doveadm backup -u user@newserver -R ssh sudouser@oldserver sudo 
/usr/bin/doveadm dsync-server -u user@oldserver

needs to be run as root on new server. Also ssh access with keys and without 
password is recommended.

Sami




Re: dsync connection issue

2018-12-28 Thread Sami Ketola



> On 27 Dec 2018, at 19.13, Subscription  wrote:
> 
> but when I try to do a backup with the following command from the old to the 
> new site
> 
> sudo doveadm -D -o imapc_user=user1@oldserver  -o imapc_password=pw-oldserver 
> backup -R -u user1@newserver imapc:
> 


Since both of your servers are running dovecot it would be probably better to 
use native doveadm protocol (over ssh pipe if required) to do the migration 
instead of using imapc.

something like:

doveadm backup -u user1@newserver -R ssh oldserver sudo /usr/bin/doveadm 
dsync-server -u user1@oldserver

allowing sudo and ssh login with keys.

Sami

Re: dovecot lmtp thinks that "disk quota exceeded" is "internal error"

2018-11-13 Thread Sami Ketola



> On 13 Nov 2018, at 21.06, Arkadiusz Miśkiewicz  wrote:
> 
> On 13/11/2018 15:54, Arkadiusz Miśkiewicz wrote:
>> 
>> 2.2.36 (not migrated to 2.3 yet) reports such problem:
>> 
>>> Nov 13 15:50:58 mbox dovecot: lmtp(xxx): session=, 
>>> Error: open(/var/mail/xxx/mailboxes.lock1bf6ad16b7b8b703) failed: Disk 
>>> quota exceeded
>>> Nov 13 15:50:58 mbox dovecot: lmtp(xxx): session=, 
>>> Error: Couldn't create mailbox list lock /var/mail/xxx/mailboxes.lock: 
>>> file_create_locked(/var/mail/xxx/mailboxes.lock) failed: safe_mkstemp(/var
>>> /mail/xxx/mailboxes.lock) failed: Disk quota exceeded
>>> Nov 13 15:50:58 mbox dovecot: lmtp(xxx): session=, 
>>> msgid=: save failed to open mailbox 
>>> INBOX.Spam: Internal error occurred. Refer to server log for more informat
>>> ion. [2018-11-13 15:50:58]
>> 
>> Looks a bug to me since disk exceeded is not a internal error. Shouldn't
>> lmtp return over quota info instead of error?
>> 
> 
> Just to confirm - dovecot 2.3.3 - the same behaviour, internal error

Dovecot can't create the lock file and it's treated as internal error. Why do 
you think that it should not be treated as such?

Sami

Re: Proxy + 2x static passdb not working as expected

2018-11-13 Thread Sami Ketola



> On 13 Nov 2018, at 12.19, Michael Goth  wrote:
> 
> Hi all,
> 
> I'm trying to setup a Dovecot proxy that authenticates the user against two 
> backend servers. If login server1 fails, server2 should be tried.
> 
> The problem: Only the first server seems to be tried, even if the login fails.
> 
> Config snippet:
> 
>  protocol imap {
>passdb {
>  driver = static
>  args = proxy=y nopassword=y host=oldserver1.example.com port=993 ssl=y
>}
> 
>passdb {
>  driver = static
>  args = proxy=y nopassword=y host=oldserver2.example.com port=993 ssl=y
>}
>  }
> 
> With this config, only accounts on oldserver1.example.com can login. If I 
> reverse the two passdb entries, only accounts on oldserver2.example.com can 
> login.
> 
> I've done the same with SQL passdb first + static passdb second. That worked 
> as expected. Not sure what I'm doing wrong here.


The config does work as expected. Static passdb with nopassword=y always 
matches and the entry is used.
You need to have some kind of key on a database to indicate correct backend and 
then build passdb -setup to 
utilise that key.

Sami

Re: error Cached MIME parts don't match message during parsing: Cached header size mismatch (parts=)

2018-11-02 Thread Sami Ketola


> On 2 Nov 2018, at 20.58, Poliman - Serwis  wrote:
> 
> I have a problem for specific mailbox. In mail.err file I see a lot:
> Nov  2 07:41:17 s1 dovecot: imap(ar...@example.pl ): 
> Error: unlink(/var/vmail/example.pl/artur/Maildir/dovecot.index.cache 
> ) failed: No such file 
> or directory (in mail-cache.c:28)
> Nov  2 07:41:17 s1 dovecot: imap(ar...@example.pl ): 
> Error: Corrupted index cache file 
> /var/vmail/example.pl/artur/Maildir/dovecot.index.cache 
> : Broken MIME parts for 
> mail UID 2356 in mailbox INBOX: Cached MIME parts don't match message during 
> parsing: Cached header size mismatch (parts=)
> Nov  2 07:41:19 s1 dovecot: imap(ar...@example.pl ): 
> Error: unlink(/var/vmail/example.pl/artur/Maildir/dovecot.index.cache 
> ) failed: No such file 
> or directory (in mail-cache.c:28)
> Nov  2 07:41:19 s1 dovecot: imap(ar...@example.pl ): 
> Error: Corrupted index cache file 
> /var/vmail/example.pl/artur/Maildir/dovecot.index.cache 
> : Broken MIME parts for 
> mail UID 2431 in mailbox INBOX: Cached MIME parts don't match message during 
> parsing: Cached header size mismatch (parts=)
> Nov  2 07:41:21 s1 dovecot: imap(ar...@example.pl ): 
> Error: unlink(/var/vmail/example.pl/artur/Maildir/dovecot.index.cache 
> ) failed: No such file 
> or directory (in mail-cache.c:28)
> Nov  2 07:41:21 s1 dovecot: imap(ar...@example.pl ): 
> Error: Corrupted index cache file 
> /var/vmail/example.pl/artur/Maildir/dovecot.index.cache 
> : Broken MIME parts for 
> mail UID 2552 in mailbox INBOX: Cached MIME parts don't match message during 
> parsing: Cached header size mismatch (parts=)
> Nov  2 07:41:21 s1 dovecot: imap(ar...@example.pl ): 
> Error: unlink(/var/vmail/example.pl/artur/Maildir/dovecot.index.cache 
> ) failed: No such file 
> or directory (in mail-cache.c:28)
> Nov  2 07:41:21 s1 dovecot: imap(ar...@example.pl ): 
> Error: Corrupted index cache file 
> /var/vmail/example.pl/artur/Maildir/dovecot.index.cache 
> : Broken MIME parts for 
> mail UID 2605 in mailbox INBOX: Cached MIME parts don't match message during 
> parsing: Cached header size mismatch (parts=)
> Nov  2 07:41:24 s1 dovecot: imap(ar...@example.pl ): 
> Error: unlink(/var/vmail/example.pl/artur/Maildir/dovecot.index.cache 
> ) failed: No such file 
> or directory (in mail-cache.c:28)
> Nov  2 07:41:24 s1 dovecot: imap(ar...@example.pl ): 
> Error: Corrupted index cache file 
> /var/vmail/example.pl/artur/Maildir/dovecot.index.cache 
> : Broken MIME parts for 
> mail UID 2717 in mailbox INBOX: Cached MIME parts don't match message during 
> parsing: Cached header size mismatch (parts=)
> I have many mailboxes on the server and each of them work perfectly except 
> this one.

Somehow cache files have been corrupted.

You can try to fix them with: doveadm force-resync -u user INBOX

Sami




Re: clients out of sync after restoring backup

2018-10-30 Thread Sami Ketola



> On 29 Oct 2018, at 22.32, Frank-Ulrich Sommer  wrote:
> 
> Hi,
> due to a failing disk I had to restore everything from a backup including 
> dovecot (2.2.22) and the Maildir containing the stored mails. A period of 
> some days must later be restored manuall, but that is beyond scope for this 
> question.
> 
> Naturally this causes confusion for the clients as they keep mails that are 
> no longer on the server. That was expected but what puzzles me is that mails 
> arriving after the succesful restore are only sporadically detected by some 
> clients (e.g. K9 and Thunderbird) that were connected before the crash. A 
> client that was connected for the first time after restoring from the backup 
> works exactly as expected.
> 
> Might this situation lead to reusing UUIDs that some clients still remember 
> as belonging to a different mail? Is there a clean solution for this on the 
> server side or is it necessary to purge locally stored data by the clients 
> (e.g. by temporarily removing account)?

You can try to change the mailbox UIDVALIDITY with doveadm mailbox update. When 
UIDVALIDITY changes clients should invalidate their caches and at minimum 
redownload all mail headers again. Some will also redownload all mail bodies.
But then there is also clients that do not obey RFC and do nothing. For those 
the only way is to temporarily remove account and reconfigure account. 

And then there is POP3 clients that require to remove and reconfigure accounts 
or the end users might see duplicate emails.

Sami



Re: Calendar function ?

2018-10-22 Thread Sami Ketola



> On 21 Oct 2018, at 16.54, Marcus Rueckert  wrote:
> 
> On 2018-10-21 13:51, m...@caloro.ch wrote:
>> I need the possibilities to Sync the calender with diffrent Clients
>> like Outlook, LotusNotes, Android, and Iphone Devices.
> 
> Well carddav/caldav can do that. For outlook there is a plugin. for android 
> you can use davdroid. iphone I dont know. might even be supported out of the 
> box?


iOS does support CalDAV natively out of the box.

Sami

Re: dovecot passdb driver=checkpassword as external script for block ip

2018-10-18 Thread Sami Ketola


> On 17 Oct 2018, at 20.09, bes  wrote:
> 
> Refer to
> https://dovecot.org/pipermail/dovecot/2015-March/099971.html
> https://wiki.dovecot.org/PasswordDatabase
> I tried to repeat the same thing.
> Set these passdb:
> 
> passdb {
>  args = /myscript.sh ip=%r
>  driver = checkpassword
>  result_failure = return-fail
>  result_success = continue
> }
> passdb {
>  args = /etc/dovecot/dovecot-sql.conf.ext ( my working auth method )
>  driver = sql
> }
> 
> Created executable /myscript.sh:
> #!/bin/sh
> exit 1


While I'm not sure why this does not work I'm was wondering if a deny-passdb 
would be more 
efficient to implement this blacklisting as executing a script on each login 
attempt is awfully slow. 

Instead I would do something like this:

  # check deny passwd for ip address first
  passdb {
driver = passwd-file
args = username_format=%r /etc/dovecot/ip-deny-list
deny = yes
auth_verbose = no
result_success = return-fail
  }

and then just create /etc/dovecot/ip-deny-list with one ip address per line.


Sami



Re: immediate delete of mails

2018-10-09 Thread Sami Ketola



> On 9 Oct 2018, at 11.19, Tamas Selmeci  wrote:
> 
> When I press DEL on a mail, the file /home/tselmeci/mail/Trash increases and 
> the deleted mail is appended to it. But it doesn't disappear from 
> /var/spool/mail/tselmeci. Once I expunge, /home/tselmeci/mail/Trash becomes 
> empty but the original mail is still present in /var/spool/mail/tselmeci.
> 
> Maybe the SMTP server holds a write lock on the spool file thus preventing 
> Dovecot from erasing the mail?

How do you deliver then mails to the server? 

also mbox is very unoptimised mailbox format for this. Each time mail is 
deleted from mbox mailbox the whole mbox while needs to be rewritten to remove 
the mail.

Sami




Re: outlook idiocy - IMAP folders with /

2018-10-02 Thread Sami Ketola


> On 2 Oct 2018, at 15.28, Jerry  wrote:
>> 
> 
> Nobody is going to stop using Outlook because some *.nix user has gotten his
> shorts up in a knot. Now, I did ask this question on a MS Outlook Tech Forum.
> One of the first responses I got was to ask exactly what Dovecot's response
> was to the attempt to create this folder; i.e., what error message or code
> was returned by Dovecot. I don't have the answer to that. If someone can
> supply me with the complete and accurate return code I will post that and see
> what transpires.
> 

A NO [CANNOT] Character not allowed in mailbox name: '/' (0.001 + 0.000 secs).

Sami



Re: Need to convert mbox to Maildir

2018-10-02 Thread Sami Ketola


> On 2 Oct 2018, at 21.05, Mark Foley  wrote:
> 
> I have a mbox file of emails.  I want to convert this to Maildir giving me 
> individual message
> files per email.  I've looked at dsync, but as far as I can tell this wants a 
> specific target
> user and it appears that it will "distribute" the converted messages into 
> that user's INBOX. 
> 
> I don't want to put these mbox messages into any particular user's Maildir 
> hierarchy, just
> export to file-per-message format to a destination directory of my choosing. 
> 
> Is this possible?


Yes. 

[root@ketola /]# mkdir /test
[root@ketola /]# chown vmail /test
[root@ketola /]# doveadm backup -u sami Maildir:/test/

done.

Sami

Re: v2.3.3 rc1 - Error: sieve: !!BUG!!: Binary compiled from dovecot.sieve is still corrupt

2018-09-27 Thread Sami Ketola



> On 27 Sep 2018, at 16.53, James  wrote:
> 
> On 27/09/2018 13:40, Stephan Bosch wrote:
> 
>>> Address   Line  Code
>>> :   DEBUG BLOCK: 3
>>> 0001:   EXTENSIONS [1]:
>>> 0002: vacation
>>> 0004:2: VACATION
>>> 0007:4:   seconds: NUM 5
>>> 0009: Binary is corrupt.
>>> 
>>> The line numbers differs and 86400 is read as 5.  It is like it has
>>> forgotten the size of an integer or is confused about endianness.
>>> There is something strange, like an #if that guesses wrong.  At least
>>> I have somewhere to start looking.
>>> 
>>> Thank you for checking at your end, I was worried the RC had
>>> introduced an error and your result suggests not.  RCs are for testing
>>> and I am.
>> 
>> The number is stored as a chain of bytes of which the most significant
>> bit indicates whether the next byte still belongs to the number. If this
>> bit is somehow interpreted wrong, the first byte of this number would
>> read as 5, thereby returning '5' as the result and ignoring subsequent
>> bytes (causing corruption at the next item to read).
>> 
>> Since you're using SunOS, your compiler may be doing something funky.
>> Which compiler is used anyway? Perhaps different versions for the
>> Dovecot releases that do and don't work?
> 
> It was studio cc.  gcc doesn't make it through configure and I didn't ask 
> why.  I have some other things to do but will look at this again later.  
> Thank you for the byte code explanations.  The coding at this point is hard 
> to follow with the pointers-to-functions and #defines.


Can you share a little bit more info on how did the compile (or configure even) 
fail with gcc on Solaris 11?

as I have no problems in compiling dovecot and pigeonhole on my Solaris 11.3 
system with gcc. The version that ships with my Solaris is 4.5.2.

I also have Sun Studio 12.5 installed but I have not even tried to compile 
dovecot wit that yet.

Sami



Re: Local access to IMAP mailboxes

2018-09-26 Thread Sami Ketola



> On 26 Sep 2018, at 13.07, Aki Tuomi  wrote:
> 
> 
> 
> On 26.09.2018 12:22, Victor Sudakov wrote:
>> Dear Colleagues,
>> 
>> I have made mailboxes in ~/Mail available via IMAP (Dovecot 2.3.2.1),
>> that is: "mail_location = mbox:~/Mail:INBOX=/var/mail/%u"
>> 
>> However, I often read and modify the mailboxes locally with mutt (e.g.
>> append and delete mails).
>> 
>> Should I expect any problems wit Dovecot indexes etc? What if I even
>> do "rm ~/Mail/some/mailbox", will Dovecot be mad at me?
>> 
> 
> Dovecot is tolerant to changes with mbox and maildir. It will reindex if
> it detects someone changing them outside.


That is true but it however has performance impacts if the caching is impaired 
but still does works.

Sami

Re: Dsync deleting mailboxes due to duplicate UIDs

2018-09-10 Thread Sami Ketola



> On 10 Sep 2018, at 0.05, Andreas Thienemann  wrote:
> 
> Following the robustness principle it feels to me that it would make sense 
> for dsync to disregard a duplicate header from a remote server and only use 
> the first occurence.
> 
> Would that be a good approach to the problem? Now that I understand the 
> problem I am having, I can just workaround it but it seems to me that dsync 
> should handle this case better.
> 

Currently dovecot does rely on remote to send valid RFC-compliant headers. But 
you are not alone, we have seen similar problems with mixed header fields on 
some legacy servers on the migrations we have performed.

I have found dsync_hashed_headers setting as a good workaround for migrating 
mails from broken imap servers. As per RFC Message-ID should be unique and we 
have safely used setting dsync_hashed_headers=Message-ID to migrate the 
problematic users away from the broken servers.

Sami



Re: Dsync deleting mailboxes due to duplicate UIDs

2018-09-09 Thread Sami Ketola



> On 9 Sep 2018, at 18.42, Andreas Thienemann  wrote:
> 
> Hi,
> 
> I am attempting to migrate a mailspool from a cyrus server to a dovecot 
> server using the dsync backup approach as described in the wiki at 
> .
> 
> The first attempt works great. Everything copies over and a quick glance over 
> the spool looks good.
> 
> Running doveadm backup again though results in the following error:
> 
> dsync(eggs): Warning: Deleting mailbox 'INBOX.MailRestore': UID=39 already 
> exists locally for a different mail: Headers hashes don't match 
> (18d567fc7e258a67e47b629c8bb16500 vs 230354b2d5cad21ebbb4a7440b977adb)
> 
> As promised, the folder MailRestore is gone after dsync finishes.
> 
> Running doveadm backup again for the third time copies the folder again.
> 
> Running it a fourth time gives the same error and the folder is gone.
> 
> Trying to figure this out I initially had a few mails where the header hashes 
> were 68b329da9893e34099c7d8ad5cb9c940, the md5sum of an empty string.
> Turns out that the source mailspool had a few broken emails and cleaning 
> these out fixed most my issues.
> 
> But in this case, I am stumped. UID 39 on the server is a valid mail with 
> correct headers and everything.
> 
> Is there a good way of identifying the two mails that seem to clash? I had a 
> quick look at the dovecot code but did not see how the header_stream gets 
> hashed into the hdr_hash used for comparing mails.


Is it possible to get imapc rawlogs to analyze? create a directory that is 
writeable and add -o imapc_rawlog_dir=/path/to/directory to command line.

Also with recent dovecot release you can tell dsync which header fields to hash 
when matching mails. Add dsync_hashed_headers=Message-ID to config and dovecot 
will only mach mails using Message-ID header fileld.

dsync_hashed_headers setting is supported since dovecot 2.2.33.

Sami

Re: Auth process sometimes stop responding after upgrade

2018-09-07 Thread Sami Ketola


> On 7 Sep 2018, at 11.25, Simone Lazzaris  wrote:
> Actually, I have a poolmon script running that should drop vhost count for 
> unresponsive backends; the strage thing is, the backends are NOT 
> unresponsive, they are working as ususal.
> 

If it's this one https://github.com/brandond/poolmon/blob/master/poolmon 
 
you are running and old version of it as the latest version is more compatible 
with recent dovecot releases.

current version in git correctly uses HOST-DOWN and HOST-FLUSH instead of 
modifying vhost count.

But even then I still consider poolmon a bit too aggressive in marking hosts 
down. It does HOST-DOWN and HOST-FLUSH already after first scan failure. 
Maybe there should be some grace period and wait for few failed polls in 
certain period before doing it in case there is some temporary overload or 
networking failure.

Sami






Re: Auth process sometimes stop responding after upgrade

2018-09-07 Thread Sami Ketola



> On 7 Sep 2018, at 11.00, Simone Lazzaris  wrote:
> 
> 
> The only suspect thing is this:
> 
> Sep  6 14:45:41 imap-front13 dovecot: director: doveadm: Host 192.168.1.142 
> vhost count changed from 100 to 0
> Sep  6 14:45:41 imap-front13 dovecot: director: doveadm: Host 192.168.1.143 
> vhost count changed from 100 to 0
> Sep  6 14:45:41 imap-front13 dovecot: director: doveadm: Host 192.168.1.219 
> vhost count changed from 100 to 0
> 
> Nothing on the other system logs (e.g. kernel, daemon, syslog, messages ).

Any idea what is changing the vhost count on the backends? Do you have some 
script running that possibly does change the vhost count for some triggering 
event?

Sami



Re: Best way to move mail from one server to another

2018-09-04 Thread Sami Ketola



> On 4 Sep 2018, at 21.24, Ruben Safir  wrote:
> 
> On 9/4/18 12:27 PM, Sami Ketola wrote:
>> 
>> 
>>> On 4 Sep 2018, at 18.38, Robert Schetterer  wrote:
>>> 
>>> Sorry i migrated terrabytes of mail with imapsync and never had a
>>> problem, it works as designed, also with maildir rsync did a good job,
>>> what never worked as it should was dsync ,cause of bugs ,that may
>>> changed now
>>> 
>> 
>> I would like to very much hear about the problems you had with dsync as we 
>> have successfully migrated tens of petabytes of mails with it successfully 
>> while preserving the UID numbers.
>> Imapsync would have been totally out of question as it is absolutely crucial 
>> in those migrations to keep the UID -> MAIL pairs matching legacy server. 
>> And usually also POP3 UIDLs. If not keeping the data the servers would have 
>> literally melted under load when switching over then of thousands of users 
>> forcing them to redownload headers of even mail bodies for millions of mails.
>> 
>> Sami
>> 
>> 
> 
> 
> why does not rsync work?

Nobody said it does not work. But it can be used only if the mail storage 
format does not change.

Sami

Re: Best way to move mail from one server to another

2018-09-04 Thread Sami Ketola


> On 4 Sep 2018, at 19.32, Rick Romero  wrote:
>> No it does not. This is different thing. This is about managing duplicates 
>> on multiple syncs.
>> Imapsync seems to keep track of UID -> MAIL mapping for server A and UID -> 
>> MAIL mapping for server B so that on multiple runs it does not sync the same 
>> messages again. However it does not use the same UID -> MAIL for server B as 
>> is originally on server A. And that is the problem.
> That's not what you said, you said there was a client sync issue.  That's 
> definitely an issue (especially if you run Mac Mail).
> 
> Ignoring duplicates on multiples runs is a non-issue, even with the 2011 
> version.  Recording UIDs is not necessary for that.
> 

Nope. I said that with imapsync there is no way to preserve the same UID for a 
mail on the destination. Data is lost.

If the UID -> MAIL pairs are not kept then client caches need to be invalidated 
and redownloaded.

Sami



Re: Best way to move mail from one server to another

2018-09-04 Thread Sami Ketola



> On 4 Sep 2018, at 18.38, Robert Schetterer  wrote:
> 
> Sorry i migrated terrabytes of mail with imapsync and never had a
> problem, it works as designed, also with maildir rsync did a good job,
> what never worked as it should was dsync ,cause of bugs ,that may
> changed now
> 

I would like to very much hear about the problems you had with dsync as we have 
successfully migrated tens of petabytes of mails with it successfully while 
preserving the UID numbers.
Imapsync would have been totally out of question as it is absolutely crucial in 
those migrations to keep the UID -> MAIL pairs matching legacy server. And 
usually also POP3 UIDLs. If not keeping the data the servers would have 
literally melted under load when switching over then of thousands of users 
forcing them to redownload headers of even mail bodies for millions of mails.

Sami




Re: Best way to move mail from one server to another

2018-09-04 Thread Sami Ketola


> On 4 Sep 2018, at 18.45, Rick Romero  wrote:
> https://imapsync.lamiral.info/FAQ.d/FAQ.Duplicates.txt
> 
> Seems to use UIDs so that 'data' isn't lost.
> 
> 
No it does not. This is different thing. This is about managing duplicates on 
multiple syncs.
Imapsync seems to keep track of UID -> MAIL mapping for server A and UID -> 
MAIL mapping for server B so that on multiple runs it does not sync the same 
messages again. However it does not use the same UID -> MAIL for server B as is 
originally on server A. And that is the problem.

Sami



Re: Best way to move mail from one server to another

2018-09-04 Thread Sami Ketola



> On 4 Sep 2018, at 18.00, Robert Schetterer  wrote:
> 
> Am 04.09.2018 um 16:52 schrieb Sami Ketola:
>> 
>> 
>>> On 4 Sep 2018, at 17.47, Robert Schetterer  wrote:
>>> 
>>> Am 04.09.2018 um 09:41 schrieb Sami Ketola:
>>>> imapsync always loses data
>>> 
>>> never saw this, be carefull by anounce such myths
>> 
>> 
>> It is a fact. Imapsync works over IMAP only and IMAP protocol does not even 
>> support transferring all data. At most at least UID numbering will be lost 
>> and end users need to invalidate their local caches.
> 
> but using "looses data" might others think it also may fail with imap,
> so be more detailed next time


UID number is also data that is saved on the IMAP backend. If that is lost then 
it's "lost data".

Sami

Re: Best way to move mail from one server to another

2018-09-04 Thread Sami Ketola



> On 4 Sep 2018, at 17.47, Robert Schetterer  wrote:
> 
> Am 04.09.2018 um 09:41 schrieb Sami Ketola:
>> imapsync always loses data
> 
> never saw this, be carefull by anounce such myths


It is a fact. Imapsync works over IMAP only and IMAP protocol does not even 
support transferring all data. At most at least UID numbering will be lost and 
end users need to invalidate their local caches.

Sami

Re: Best way to move mail from one server to another

2018-09-04 Thread Sami Ketola



> On 4 Sep 2018, at 10.20, James Brown  wrote:
> 
> I’d like to migrate to a new server. I only need to transfer the mail store, 
> have already done the users. I’ve seen different recommendations:
> 
> IMAP-Sync
> Rsync
> Doveadmin backup and
> dsync
> 
> What are the pros and cons of each? What would be best for me? I’m thinking 
> that Doveadmin backup is the way to go.
> 
> Both old and new servers are on same network. New server is running Dovecot 
> 2.3.2.1, old server is 2.2.32. macOS X.
> 
> Does anyone have any advice or examples?


It depends. Are your users already receiving mails to the new server? If yes, 
then the only way is to use "doveadm sync -1" or data will be lost. Even then 
data may be lost as UID numbers might already be used in the new server.
If your users are not receiving mails to the new server it still depends. Is 
the mail storage format same in both servers? If yes, then it's probably 
fastest to use rsync. If not, then you need to use doveadm backup.

imapsync always loses data.

Sami



Re: online conversion using replication?

2018-09-03 Thread Sami Ketola


> On 3 Sep 2018, at 21.49, B. Reino  wrote:
> 
> On Mon, 3 Sep 2018, Sami Ketola wrote:
> 
>>> On 3 Sep 2018, at 4.18, Daniel Miller  wrote:
>>> 
>>> That works for a one-time migration, or perhaps via a cron-job, but what I 
>>> want is basically a constant one-way backup and it seems replication could 
>>> do it more elegantly & efficiently.
>>> 
>> 
>> So you want real-time archiving? What we have done with couple of customers 
>> is that we just configure MTA to replicate all incoming mails to secondary 
>> site.
> 
> Would you mind showing how you're doing it?
> (hopefully with postfix, otherwise it may not be so interesting to me..)
> 

Those customers were not using Postfix but I believe that with 
http://www.postfix.org/postconf.5.html#recipient_bcc_maps 
<http://www.postfix.org/postconf.5.html#recipient_bcc_maps> similar behaviour 
can be achieved with Postfix. 

Sami




Re: online conversion using replication?

2018-09-03 Thread Sami Ketola


> On 3 Sep 2018, at 4.18, Daniel Miller  wrote:
> 
> That works for a one-time migration, or perhaps via a cron-job, but what I 
> want is basically a constant one-way backup and it seems replication could do 
> it more elegantly & efficiently.
> 

So you want real-time archiving? What we have done with couple of customers is 
that we just configure MTA to replicate all incoming mails to secondary site.

Sami



Re: Is the Doveadm HTTP API considered stable for production use?

2018-08-23 Thread Sami Ketola



> On 22 Aug 2018, at 18.55, James Beck  wrote:
> 
> Hi,
> 
> I'm running 2.2.34 in production (installed from Debian stretch
> backports) and want to rework some scripts. Can the HTTP API be
> considered stable in 2.2.34 please? The wiki says it is "considered
> experimental in v2.2.22" so I thought I'd check before writing API calls
> rather than a wrapper around doveadm!


I think it can be now considered stable. We even have couple of environments
where it's actively used in production. 

Sami


Re: Imap special-use with IOS 11

2018-08-17 Thread Sami Ketola

> On 17 Aug 2018, at 17.20, Federico Bartolucci  wrote:
> 
> Hello,
> 
> has anyone experiened any issues with IMAP SPECIAL-USE extension with ios11?
> After the update to ios11 the Ios mail client seems to not recognize anymore 
> the imap special-use, so that system folders (Sent, Trash etc..) are not 
> authomatically recognized and set during the configuration.

Can you please verify that your dovecot advertises in the pre-login or 
post-login banner that SPECIAL-USE is advertised to be supported?
iOS Mail.app does not seem to use the special use flags unless they are 
advertised to be supported. (Just like RFC mandates)

Sami




Re: doveadm mailbox delete not working

2018-08-15 Thread Sami Ketola


> On 15 Aug 2018, at 9.29, Ralf Becker  wrote:
> 
> Am 14.08.18 um 18:51 schrieb Aki Tuomi:
>> Try 
>> 
>> doveadm mailbox list -u user INBOX/*
> 
> Hmm, posted that before, it lists all these undeletable mailboxes:


Can you also post your doveconf -n to be sure that the folder is just not 
autocreated after delete.

Sami

Re: limit sharing ability to certain users

2018-08-10 Thread Sami Ketola



> On 9 Aug 2018, at 8.48, Simeon Ott  wrote:
> 
> Thanks Sami, thanks Aki
> 
> I just updated the packages on our testing server and now it works like 
> expected.
> There are some LDAP tests to come. Are there many productive server out there 
> using this repository?

We have no way of knowing that. We could probably check the logs for the number 
how many times the 
packages have been downloaded, but even then we don't know for what purpose 
those packages are 
downloaded for. 

Sami



Re: limit sharing ability to certain users

2018-08-08 Thread Sami Ketola

http://repo.dovecot.org/ 

Sami


> On 8 Aug 2018, at 10.27, Simeon Ott  wrote:
> 
> Okay, this seems to be due to the fact that the option “use_globals_only" is 
> supported only in v2.2.31+
> We are on Debian jessie with dovecot v2.2.13 – even an upgrade to current 
> stable stretch won’t help (dovecot v2.2.27). So we will wait until the 
> packages find their way into the repository.
> 
> thanks anyway
> 
> 
>> On 7 Aug 2018, at 13:00, Simeon Ott > > wrote:
>> but, did you read my last note anyway?
>> IMPORTANT NOTE: anyway.. even with this options set (acl and 
>> acl_globals_only) the user t...@onnet.ch  is still 
>> able to share its own folders?!
>> 
>> root@buserver:/etc/dovecot# doveadm user t...@onnet.ch 
>> fieldvalue
>> uid  5000
>> gid  5000
>> home /var/spool/postfix/virtual/onnet.ch/test/ 
>> mail maildir:~/Maildir
>> quota_rule   *:bytes=1073741824
>> acl  vfile:/etc/dovecot/dovecot-acl
>> acl_globals_only yes
>> 
>> root@buserver:/etc/dovecot# telnet localhost 143
>> Trying ::1...
>> Connected to localhost.
>> Escape character is '^]'.
>> * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE 
>> AUTH=PLAIN AUTH=LOGIN] Dovecot ready.
>> . login t...@onnet.ch  *
>> . OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE 
>> SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT 
>> MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS 
>> LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN 
>> CONTEXT=SEARCH LIST-STATUS SPECIAL-USE BINARY MOVE QUOTA ACL RIGHTS=texk] 
>> Logged in
>> . SETACL Inbox te...@onnet.ch  lrwstipekxa
>> . OK Setacl complete.
>> . GETACL Inbox
>> * ACL Inbox te...@onnet.ch  akxeilprwtscd 
>> t...@onnet.ch  lrwstipekxacd
>> . OK Getacl completed.
> 



Re: limit sharing ability to certain users

2018-08-06 Thread Sami Ketola


> On 6 Aug 2018, at 8.26, Aki Tuomi  wrote:
> 
> You could do
> 
> userdb {
>driver = username_format=%Lu passwd-file
>args = /etc/dovecot/share.passwd
> }
> 
> #  /etc/dovecot/share.passwd
> t...@onnet.ch ::: 
> userdb_acl=vfile:/etc/dovecot/dovecot-acl userdb_acl_globals_only = yes
> 
> should prevent the user from modifying any ACL files. 
> 

close, but few typos. userdb should be:

userdb {
  driver = passwd-file
  args = username_format=%Lu /etc/dovecot/share.passwd
}

Sami




Re: Dbox lost messages

2018-07-20 Thread Sami Ketola


> On 20 Jul 2018, at 15.20, Fineware PM  wrote:
> 
> if I dump the index of inbox :
> "doveadm -v dump example.com/user/idx/mailboxes/INBOX/dovecot.index" 
>  
> I don't find the message.
> I think is corrupted the index
> 
> 


try then running: doveadm force-resync -u user "*"

Sami

Re: Dbox lost messages

2018-07-20 Thread Sami Ketola


> On 20 Jul 2018, at 14.48, Fineware PM  wrote:
> 
> Hello,
> 
> I use dovecot with dbox storage system.
> 
> Today I receive 2 mail (I see that in the maillog ) but I don't find in INBOX 
> using my IMAP client.
> 
> I look in the files m.* and I find the message that I receive. But I cannot 
> find it in any IMAP folder.
> 
> I try to run "doveadm force-resync -u user inbox" but nothing change.
> 
> I export all email using the command "doveadm backup -u user 
> mbox:/tmp/mail:INDEX:/tmp/imap:LAYOUT:maildir++
> 
> and in the mbox file for inbox I found the 2 mails arrived.
> 
> How I can understand why I cannot see the messages ? or how I can recover it ?
> 
> Bet regards.
> 
> 

Can you see anything in the logs related? Do you use sieve scripts? Does the 
user have sieve script that might have moved/deleted the message?
Do you have maillog plugin enabled to log what the users do? Can you see the 
user deleting the mails? https://wiki.dovecot.org/Plugins/MailLog 


Sami




Re: OT: 'lost' emails

2018-07-20 Thread Sami Ketola



> On 20 Jul 2018, at 13.31, Voytek  wrote:
> 
> I suspect this is mail client issue, but looking for any suggestions:
> 
> user with Thunderbird says " I'm losing emails, mail is not in inbox, but, 
> when I search, that email shows in search result, but, I can't open the email 
> body" i don't fully understand what he was trying to say, and, won't be able 
> to look at his PC till Monday.
> 
> Looking at inbox's 'cur' directory, I can see the'missing' email amongst the 
> other 2000 emails in 'cur'
> 
> Is there any indexing I should do on the user's email folder, how?
> 
> or, is this purely Thunderbird issue, and, I need to re index Thunderbird?

https://support.mozilla.org/it/questions/1050806

seems so.

Sami

Re: folders not visible on copied mail folders

2018-07-18 Thread Sami Ketola



> On 19 Jul 2018, at 7.29, Mark Foley  wrote:
> 
> # doveadm mailbox list -u "mpress" | grep Dennis
> Dennis Email
> Dennis Email.Deleted Items
> Dennis Email.Deleted Items.Sent
> 
>> Then make sure that each of the three folders contain the cur, new, tmp 
>> subfolders.
> 
> They do:
> 
> drwx-- 2 mpress domusers 4096 2018-07-13 14:30 ./.Dennis\ Email/cur/
> drwx-- 2 mpress domusers 4096 2018-07-13 14:30 ./.Dennis\ Email/new/
> drwx-- 2 mpress domusers 4096 2018-07-13 14:30 ./.Dennis\ Email/tmp/
> drwx-- 2 mpress domusers 2678784 2017-06-23 12:14 ./.Dennis\ 
> Email.Deleted\ Items/cur/
> drwx-- 2 mpress domusers 180224 2016-06-20 16:29 ./.Dennis\ 
> Email.Deleted\ Items/new/
> drwx-- 2 mpress domusers 16384 2016-06-20 16:29 ./.Dennis\ Email.Deleted\ 
> Items/tmp/
> drwx-- 2 mpress domusers 507904 2018-07-13 13:02 ./.Dennis\ 
> Email.Deleted\ Items.Sent/cur/
> drwx-- 2 mpress domusers 4096 2015-02-23 17:51 ./.Dennis\ Email.Deleted\ 
> Items.Sent/new/
> drwx-- 2 mpress domusers 4096 2018-07-13 13:02 ./.Dennis\ Email.Deleted\ 
> Items.Sent/tmp/
> 

Is this possibly some kind of SELinux problem? Do you have it enabled? And if 
you do then 
did you perhaps move the Dennis's folders to new location with  mv causing the 
security context to be lost? 
What if you run restorecon on all the files?

Sami



Re: dsync panic

2018-07-12 Thread Sami Ketola


> On 7 Jul 2018, at 4.31, Infoomatic  wrote:
> 
> Hi,
> I just upgraded from dovecot 2.2.19 to 2.3.2. "doveadm backup" worked fine in 
> v 2.2.19, but now panics (user with shared folder):
> /opt/dovecot/bin/doveadm backup -u testuser -1 sdbox:/tmp/testuser

Not sure why it panics, but at least your command line looks incorrect. 

please try: doveadm backup -u testuser sdbox:/tmp/testuser

your options are:

doveadm backup
doveadm sync
doveadm sync -1

https://wiki2.dovecot.org/Tools/Doveadm/Sync 


Sami

Re: manage/sieve and mysql: Storage does not support write access

2018-06-21 Thread Sami Ketola



> On 21 Jun 2018, at 12.53, Markus Raps  wrote:
> 
> Hi there,
> 
> i want to set up sieve to store the scripts in the mysql database.
> 
> through mysql logging i can see that the incoming mail triggers a search in 
> the script database
> so the mysql connection basically works.
> 
> But i cant create any script through roundcube or telnet via managesieve
> 
> Jun 21 11:48:06 srv-www-01 dovecot: managesieve-login: Login: 
> user=, method=PLAIN, rip=::1, lip=::1, mpid=31995, 
> secured, session=
> Jun 21 11:48:06 srv-www-01 dovecot: managesieve(m.r...@rapsplace.de): Error: 
> sieve: dict storage: Storage does not support write access
> Jun 21 11:48:06 srv-www-01 dovecot: managesieve(m.r...@rapsplace.de): Fatal: 
> Failed to open Sieve storage.
> 

I think the problem is that the sql-dict-sieve does not support updates. It's 
read only.

Sami



Re: After logrotation Dovecot still writes to old log file

2018-06-14 Thread Sami Ketola


> On 15 Jun 2018, at 5.17, Michael Heuberger 
>  wrote:
> 
> Thanks man
> 
> You suggesting the same as https://wiki2.dovecot.org/Logging 
>  says?
> 
> postrotate
>  kill -s 0 `cat /var/run/dovecot/master.pid` || kill -s USR1 `cat 
> /var/run/dovecot/master.pid`
> endscript
> Well, I already tried that and didn't work. Hence my guess that it's 
> something else.

Does '/usr/bin/doveadm log reopen' work?

Sami



Re: Sieve_default

2018-06-13 Thread Sami Ketola


> On 13 Jun 2018, at 20.26, @lbutlr  wrote:
> 
> 
> 
>> On 12 Jun 2018, at 01:23, Alex JOST  wrote:
>> 
>> Am 12.06.2018 um 01:01 schrieb @lbutlr:
>>> I created a sieve_default to move any spam-tagged messages into the Junk 
>>> mailbox automatically, but it doesn’t appear to be working (though the 
>>> other imapsieve_mailbox… scripts are working).
>>> From doveconf -n
>>> plugin {
>>>  imapsieve_mailbox1_before = file:/usr/lib/dovecot/sieve/report-spam.sieve
>>>  imapsieve_mailbox1_causes = COPY
>>>  imapsieve_mailbox1_name = Junk
>>>  imapsieve_mailbox2_before = file:/usr/lib/dovecot/sieve/report-ham.sieve
>>>  imapsieve_mailbox2_causes = COPY
>>>  imapsieve_mailbox2_from = Junk
>>>  imapsieve_mailbox2_name = *
>>>  imapsieve_mailbox3_before = file:/usr/lib/dovecot/sieve/mark-read.sieve
>>>  imapsieve_mailbox3_causes = COPY
>>>  imapsieve_mailbox3_name = Archive
>>>  sieve = file:~/.sieve;active=~/.active_sieve
>>>  sieve_default = /usr/lib/dovecot/sieve/default.sieve
>>>  sieve_default_name = spamassassin
>>>  sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment
>>>  sieve_pipe_bin_dir = /usr/lib/dovecot/sieve
>>>  sieve_plugins = sieve_imapsieve sieve_extprograms
>>> }
>>> And
>>> # cat /usr/lib/dovecot/sieve/default.sieve
>>> if header :contains "X-spam-flag" "YES" {
>>> fileinto "Junk";
>>> }
>>> There are no user sieve files
>>> # find /home -name "*sieve*" -type f
>>> #
>>> The x-spam-flag header in the message in my mailbox is definitely there and 
>>> definitely set to yes.
>> 
>> I think you need to enable the 'fileinto' extension via 'require'. And did 
>> you pre-compile the script?
> 
> I added 
> 
> require "fileinto”;
> 
> To default.sieve
> 
> I did not pre-compile the script, but according to the docs that should not 
> prevent it from working.
> 
> Spam, messages are still not being filed in to Junk, however.


Are you sure your sieve even gets executed? Anything in the logs? can you 
enable mail_debug=yes and try again?
Can you post your doveconf -n?

Sami



Re: Autofile Junk for all users

2018-05-23 Thread Sami Ketola


> On 23 May 2018, at 16.17, André Rodier  wrote:
> 
> On 2018-05-23 15:11, @lbutlr wrote:
>> How would I get the rule
>> if header :contains "X-spam-flag" "YES" {
>> fileinto "Junk";
>> }
>> To apply to all users mail at delivery time?
> 
> Use the default scripts, works well for me.
> 
> https://wiki2.dovecot.org/Pigeonhole/Sieve/Configuration#Visible_Default_Script
> 
> The Dovecot wiki is really helpful, have a look on it ;-)


Actually in this case it's probably better to use sieve_before setting for the 
spam filter, as users are allowed to overwrite default sieve.

Sami



Re: Disconnecting unauthenticated IMAP entities faster?

2018-05-18 Thread Sami Ketola


> On 18 May 2018, at 20.19, David Hubbard  wrote:
> 
> Hello, given the 2015 revision date, I was curious if anyone can confirm 
> https://wiki2.dovecot.org/Timeouts is still accurate where the 'before login' 
> IMAP timeout remains hard coded?
> 
> We're having an issue where blocks of IP's from China and similar locations 
> are crawling IP ranges trying common login credentials, and hanging the 
> connections open in the process.  We have clients who have large numbers of 
> employees at single locations, so it isn't possible to reduce the 
> mail_max_userip_connections (assuming it even applies to pre-auth sessions) 
> to a low value.  The end result is these connections chew up all the 
> imap-login processes because they sit there until the three-minute timeout is 
> hit, blocking legit users.  The only workaround is to raise both the imap and 
> imap-login processes to a massive amount to support all the pre-auth hung 
> open connections.
> 
> It would be a lot easier to find a reasonable process limit if we could boot 
> these unauthenticated connections off in a more reasonable amount of time, 
> like 5-10 seconds, but I'm not seeing a way to accomplish that?
> 

https://github.com/PowerDNS/weakforced  
is just for situations like this.

Sami

Re: External Program for Authentication?

2018-05-16 Thread Sami Ketola


> On 16 May 2018, at 22.18, Marc Perkel  wrote:
> 
> Is it possible to run a bash script for authentication where a 0 exit code 
> indicates success and a non-zero is failure? What I'm trying to do is create 
> a shadow IMAP server that authenticates against a different server. That way 
> my server will use the same passwords as an existing server.
> 
> So what I would need is for dovecot to pass the username and password to my 
> script, I attempt to log in remotely and if I succeed I allow access on my 
> side. My side will be used to configure black lists and where spam is dragged 
> from their side to my side. (I'm a spam filtering company)
> 


Lua passdb https://wiki.dovecot.org/AuthDatabase/Lua 
 on dovecot 2.3 can be used or if 
you can make your external server act as auth policy server, then you could use 
https://wiki.dovecot.org/Authentication/Policy 
 available since dovecot 2.2.25

Sami



Re: Maillog warning

2018-05-16 Thread Sami Ketola


> On 16 May 2018, at 20.16, for...@mehl-family.fr wrote:
> 
> I have comment the line...
> 
> But now I see other warning :
> 
> Warning: /etc/dovecot/conf.d/OLD-90-quota.conf line 39: Global setting 
> mail_plugins won't change the setting inside an earlier filter at 
> /etc/dovecot/conf.d/15-lda.conf line 13 (if this is intentional, avoid this 
> warning by moving the global setting before /etc/dovecot/conf.d/15-lda.conf 
> line 13)
> 
> AND... I see an error message :
> 
> Error: Couldn't load required plugin 
> /usr/lib/dovecot/modules/lib11_imap_quota_plugin.so: Plugin quota must be 
> loaded also (you must set: mail_plugins=$mail_plugins quota)
> 
Load your global plugins first, in global config file and protocol specific 
plugins later in the config.

Sami

Re: Maillog warning

2018-05-16 Thread Sami Ketola

> On 16 May 2018, at 19.49, for...@mehl-family.fr wrote:
> 
> Hi,
> 
> In my maillog I see this warning message at every reception of e-mail:
> 
> status=sent (delivered via dovecot service (doveconf: Warning: 
> /etc/dovecot/conf.d/90-quota.conf line 39: Global setting mail_plugins won't 
> chan.
> 
> It is just a warning, the e-mail was delivered.
> 
> I don't find where is the problem.
> 
> 

You have mail_plugins setting in 90-quota.conf that is ignored because you have 
defined mail_plugins in protocol lmtp {} before that in the config. The order 
matters.

Sami



Re: dsync migration from maildir layout=fs to maildir seperator / - folders or inbox but not both

2018-05-13 Thread Sami Ketola

> On 13 May 2018, at 14.07, Matt Bryant  wrote:
> 
> Sigh ... ok so had if statement in SQL land that appended LAYOUT=fs if 
> Maildir which explains the non-converting folder layout but NOT why the 
> process is not picking up the actual contents of INBOX ...
> 

Try setting imapc_prefix=INBOX instead of imapc_list_prefix.

Sami



Re: dovecot vs. mutt: no full index sync on Maildir new/ mtime change

2018-04-24 Thread Sami Ketola


> On 24 Apr 2018, at 10.33, Michael Büker  wrote:
> 
> Hi, everyone!
> 
> This is a follow-up to "Looks like a bug to me: Dovecot ignores Maildir/new 
> timestamp" from Fredrik Roubert on 01.12.2015:
> https://www.dovecot.org/list/dovecot/2015-December/102585.html
> 
> I've run into the same problem as Fredrik: When manipulating my Maildir 
> locally with mutt, deleting a message from new/ doesn't cause a full update 
> of the index. Therefore, IMAP clients still see the deleted message.
> 
> I've read and understood Timo's reply saying that dovecot only performs a 
> "partial sync" of the index when the mtime of new/, but not of cur/, changes. 
> This makes perfect sense for performance reasons for most users:
> https://www.dovecot.org/list/dovecot/2015-December/102588.html
> 
> I, however, would be willing to take the performance hit of a full index sync 
> whenever the mtime of new/ changes. Therefore, I looked at the code and tried 
> to implement a config option (maildir_fullsync_on_new_mtime_change) for this 
> behavior. However, my understanding of 
> src/lib-storage/index/maildir/maildir-sync.c was not good enough – I probably 
> put the ctx->mbox->storage->set->maildir_fullsync_on_new_mtime_change check 
> in the wrong place, and all my patch did was ruin the index ;)
> 
> So, to summarize my question: I'd like dovecot to perform a full index sync 
> when the mtime of a Maildir's new/ has changed. I'm willing to take the 
> performance hit, because it would fix a problem I'm having with using mutt 
> and dovecot together. Can this be done in principle by adding a config option 
> check like ctx->mbox->storage->set->maildir_fullsync_on_new_mtime_change in 
> the right place in src/lib-storage/index/maildir/maildir-sync.c? If so, where 
> should it be put?


While this is probably doable with some code changes I personally "solved" the 
problem just by switching to IMAP for mutt.

Sami



Re: smtp crashing with "out of memory" for ONE particular user

2018-04-24 Thread Sami Ketola


> On 24 Apr 2018, at 9.08, Jacek Osiecki  wrote:
> 
> Finally, 512MB helped. But it is quite strange to me… Why one particular 
> mailbox, what could be causing it?


Number of mails in INBOX.

Sami



Re: [bug] Auth cache, proxy and ITERINDEX

2018-04-16 Thread Sami Ketola
Hi,

please post your doveconf -n

Sami


> On 16 Apr 2018, at 20.17, azu...@pobox.sk wrote:
> 
> Hi,
> 
> today we have exceprienced really major difficulties with our proxy backend 
> servers. Everything started after enabling auth cache:
> auth_cache_size = 100M
> auth_cache_verify_password_with_worker = yes
> auth_cache_ttl = 1 hour
> auth_cache_negative_ttl = 1 hour
> 
> Soon after we start receiving lots of calls from our customers about 'missing 
> all e-mail messages'. This was in errors logs on master server:
> 
> Apr 16 14:37:10 server00 dovecot: imap(): Error: 
> autoexpunge: Couldn't create dovecot.autoexpunge.lock lock: 
> file_create_locked(/var/mail/vhosts///home/dovecot.autoexpunge.lock)
>  failed: 
> safe_mkstemp(/var/mail/vhosts///home/dovecot.autoexpunge.lock)
>  failed: No such file or directory
> 
> Looks like that all users, who are placed on proxy backends, were logged 
> correctly BUT not proxied to the right server so they saw empty mailboxes 
> (Dovecot also created some directories on master server). What is worse, 
> their email software deleted all local emails so they are now downloading 
> everything from scratch (100Gs of mails so it will take some time).
> 
> After auth cache was disabled, things started to go to normal EXCEPT users 
> were reporting they don't see all folders - which was true, only few on them 
> were visible (INBOX, trash, sent etc.). We tried everything to make them 
> visible again, for example deleting indexes, but nothing helped. Finally, 
> removing option ITERINDEX from mail_location helped.
> 
> Can anyone explain what happened? Thank you
> 
> azurIt
> 
> 



Re: Saving Drafts with Roundcube marks old Mails in INBOX as unread.

2018-04-04 Thread Sami Ketola


> On 4 Apr 2018, at 10.59, s...@tyrion.de wrote:
> 
> Hello List,
> 
> 
> I have a problem with the client Roundcube. Whenever I save a draft to the 
> Folder INBOX.Drafts a old mail in my INBOX will be marked as unread.
> 
> Here are the IMAP Logs from Roundcube and Dovecot debug logs. I have no idea 
> why it happens. The IMAP protocol looks normal to me but as you can see in 
> the second to last line in the Dovecot log Dovecot is changing the flag of an 
> old E-Mail in INBOX that has the same UID as the Mail that was just created 
> in INBOX.Drafts.
> 
> 
> [03-Apr-2018 09:50:03 +0200]: <6c472a00> [8095] S: A0006 OK [APPENDUID 
> 1506921305 58] Append completed (0.001 + 0.042 secs).
> [03-Apr-2018 09:50:03 +0200]: <6c472a00> [8095] C: A0007 SELECT INBOX

Here roundcube SELECTs INBOX again

> [03-Apr-2018 09:50:03 +0200]: <6c472a00> [8095] S: * FLAGS (\Answered 
> \Flagged \Deleted \Seen \Draft Junk $MDNSent)
> [03-Apr-2018 09:50:03 +0200]: <6c472a00> [8095] S: * OK [PERMANENTFLAGS 
> (\Answered \Flagged \Deleted \Seen \Draft Junk $MDNSent \*)] Flags permitted.
> [03-Apr-2018 09:50:03 +0200]: <6c472a00> [8095] S: * 682 EXISTS
> [03-Apr-2018 09:50:03 +0200]: <6c472a00> [8095] S: * 0 RECENT
> [03-Apr-2018 09:50:03 +0200]: <6c472a00> [8095] S: * OK [UNSEEN 48] First 
> unseen.
> [03-Apr-2018 09:50:03 +0200]: <6c472a00> [8095] S: * OK [UIDVALIDITY 
> 1506921304] UIDs valid
> [03-Apr-2018 09:50:03 +0200]: <6c472a00> [8095] S: * OK [UIDNEXT 881] 
> Predicted next UID
> [03-Apr-2018 09:50:03 +0200]: <6c472a00> [8095] S: A0007 OK [READ-WRITE] 
> Select completed (0.000 + 0.000 secs).
> [03-Apr-2018 09:50:03 +0200]: <6c472a00> [8095] C: A0008 UID STORE 58 
> -FLAGS.SILENT (\Seen)

And here roundcube removes \Seen flag from mail UID 58

so it's roundcube that removes the \Seen flag, effectively making the mail 
unseen again. Seems like roundcube bug.

Sami



Re: Auth SEGV on sparc64, alignment problem?

2018-02-22 Thread Sami Ketola
> As this is a sparc64, with 8-byte alignment requirements, I’m guessing that’s 
> the issue.  Many a piece of software has failed to respect that and crashed.  
> But, I’m not sure.
> 
>  Does anyone have any suggestions?  I’ve built it locally (via ports), so if 
> there are compiler options I can/should try, I certainly can try.
> 
>  Thanks…

On what specific hardware you are running FreeBSD/sparc64? I have some old Sun 
desktops 
lying around with UltraSPARC-III and UltraSPARC-IIIi processors. Maybe I need 
to power them 
up again so that we can run some tests on big-endian machine ourself.

Sami



Re: Using director but stick user to a fixed backend

2018-02-17 Thread Sami Ketola


> On 17 Feb 2018, at 16.57, p...@wk-serv.de wrote:
> 
> Hi everyone,
> 
> I am using dovecot with a director setup for two backends.
> 
> I want to add a third backend for use with the apple imap push plugin.
> 
> Is there a way I can stick some users to this third backend and not using the 
> director feature?

Yes. Check Director tags introduced in release 2.2.16 
https://wiki2.dovecot.org/Director

Sami




Re: Director & Master Users

2018-02-15 Thread Sami Ketola


> On 15 Feb 2018, at 22.16, Travis Dolan  wrote:
> 
> It would look as though the changes have now negatively affected a "normal" 
> user from logging in.
> 
> 
> telnet host 143
> 
> a login username password
> 
> 
> a NO [AUTHENTICATIONFAILED] Authentication failed.
> 
> 
> telnet host 143
> 
> 1 login devteam*masteru...@example.com password
> 
> 
> 1 OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE 
> SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT 
> MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS 
> LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN 
> CONTEXT=SEARCH LIST-STATUS BINARY MOVE QUOTA] Logged in
> 
> 
> What do you think?
> 

So your director is the first entry point where the end users connect?

in that case your director should have passdb setup that verifies the user 
password and then 
switches the session to use master password when forwarding the connection to 
backend.

something like this in director:

passdb {
  driver = passwd-file
  args = /data/mail.passwd
  result_success = continue-ok
}

passdb {
  driver = static
  args = pass=masterpassword 
  skip = unauthenticated
}


and in backend:

passdb {
  driver = static
  args = password=masterpassword
}

Sami




Re: Does Dovecot LDAP auth support LDAP referral

2018-02-02 Thread Sami Ketola


> On 2 Feb 2018, at 10.38, Xuan Jia  wrote:
> 
> We using Dovecot with LDAP.
> From the beginning, we using GC LDAP query with port 3268 for email accounts.
> For example, us...@our-organization.org (in the USA) with "base = 
> dc=our-organization, dc=org" works fine.
> 
> But refer to this document:
> https://wiki2.dovecot.org/AuthDatabase/LDAP
> When we change the LDAP from 3268 to 389 and with TLS, the base should be 
> changed like this:
> "base = ou=usa, dc=our-organization, dc=org"
> 
> But if the user (user2) located in United Kingdom (ou=gbr), the user can not 
> login.
> 
> When we debug with ldapsearch:
> ldapsearch -ZZ -v -h dc.our-organization.org -p 389 -D 
> 'cn=auth_user,ou=usa,dc=our-organization,dc=org' -W -b 'dc=our-organization, 
> dc=org' '(userPrincipalName=us...@our-organization.org)'
> It can return user2 information with some "numReferences".
> 
> But in Dovecot, if "base = dc=our-organization, dc=org" it only reported auth 
> error with timeout.
> 
> So my question is: does Dovecot LDAP auth support LDAP referral?
> If Dovecot relies on OpenLDAP, it should be support.
> If not, why and what is the walkthrough?


TBH, I don't think that it's supported. Looking at the source code at least it 
does not look it is.

What you could do is to have separate passdb for both ldap bases.
one that would query base = ou=usa, dc=our-organization, dc=org and one that 
would query 
base = ou=gbr, dc=our-organization, dc=org

and then use skip=authenticated on the second passdb if user already found in 
first passdb.

Sami

Re: Live Import of IMAP

2018-02-01 Thread Sami Ketola


> On 1 Feb 2018, at 17.10, Sergio Belkin <seb...@gmail.com> wrote:
> 
> 
> 2018-02-01 11:59 GMT-03:00 Sami Ketola <sami.ket...@dovecot.fi 
> <mailto:sami.ket...@dovecot.fi>>:
> 
> 
>> On 29 Jan 2018, at 21.57, Sergio Belkin <seb...@gmail.com 
>> <mailto:seb...@gmail.com>> wrote:
>> 
>> Hi,
>> 
>> I have the mailbox of jon@example.com <mailto:jon@example.com> just 
>> migrated only with mails after migration, is it ok to run:
>> 
>> doveadm -o mail_fsync=never sync -1 -R -u j...@example.com 
>> <mailto:j...@example.com> imapc:
>> 
>> wiil that import all mails missing mails to the new server? I don't want 
>> that delete any current mail, is that right the command above?
>> 
>> thanks in advance!
> 
> Yes, if you have imapc settings defined somewhere in your config. If not you 
> need to add them to the command line.
> 
> Sami
> 
> 

Oh yeah, and you might want to add -o mail_prefetch_count=20 or maybe even more 
to speed up the migration.

Sami



Re: Live Import of IMAP

2018-02-01 Thread Sami Ketola


> On 29 Jan 2018, at 21.57, Sergio Belkin  wrote:
> 
> Hi,
> 
> I have the mailbox of jon@example.com  just 
> migrated only with mails after migration, is it ok to run:
> 
> doveadm -o mail_fsync=never sync -1 -R -u j...@example.com 
>  imapc:
> 
> wiil that import all mails missing mails to the new server? I don't want that 
> delete any current mail, is that right the command above?
> 
> thanks in advance!

Yes, if you have imapc settings defined somewhere in your config. If not you 
need to add them to the command line.

Sami




Re: Change destination of dsync

2018-01-28 Thread Sami Ketola


> On 25 Jan 2018, at 19.11, Sergio Belkin  wrote:
> 
> Hi,
> 
> I have the following command for retrieving mails from another IMAP mail 
> server
> 
> dsync -D -v backup -R -u jon.doe imapc: 
> 
> I't ok and works, but I'd want to copy messages outside of /var directory, is 
> there a way to do that?

You can always override settings on command line, such as mail_location in this 
case. just add 
-o mail_location=Maildir:/alternate/location/path

or what ever mailbox format you are using.

Sami



Re: Quota plugin creates IMAP folder

2018-01-24 Thread Sami Ketola


> On 24 Jan 2018, at 13.21, Thorsten Hater  wrote:
> 
> This is from my config
> 
> mail_location = maildir:~/Maildir
> 
> and in my userdb-ldap.conf
> 
> user_attrs = \
>   ... \
>   =home=%{ldap:homedir}, \
>   ...
> 
> Am I missing something?

So what does "doveadm user " then return as home?

Sami



  1   2   >