Re: Bug: Shared Mailbox - Case Sensitivity

2016-09-30 Thread Aki Tuomi 
Can you provide doveconf -n?


---Aki TuomiDovecot oy
 Original message From: Leander Schäfer  
Date: 01/10/2016  03:41  (GMT+02:00) To: Aki Tuomi , 
Dovecot Mailing List , Timo Sirainen  
Subject: Re: Bug: Shared Mailbox - Case Sensitivity 
Am I missing something, or might this be a bug as it seems to me?


Am 16.09.16 um 14:21 schrieb Leander Schäfer:
> Hi Aki,
>
>
> Thanks for your advice. Yes, I'm aware of this. Yet lowercasing should 
> be the default since Dovecot 2.1.x., isn't it? Yet I wouldn't know 
> where exactly to implement this %L, since the ACLs are set through 
> IMAP commands through the users mailclient like Thunderbird. So in 
> other words, the email address to whom the user want to grant ACLs 
> provided by the user's mailclient, has nothing to do with my auth 
> backend where e.g. %u => %Lu would apply. PLease correct me if I'm 
> wrong here.
>
>
> It clearly looks like a bug of the internal processing of the 
> "dovecot-acl-list" files. It simply lacks on a lowercase enforcement 
> in the code, like it already seems to do for the "dovecot-acl" file.
>
>
> Best regards
>
> Leander Schäfer
>
>
>
> Am 16.09.16 um 12:53 schrieb Aki Tuomi:
>>
>> On 16.09.2016 12:54, Leander Schäfer wrote:
>>> Hi,
>>>
>>> unfortunately I found a bug in Dovecot's ACL handling for shared
>>> mailboxes. It turns out Dovecot doesn't enforce lower casing the
>>> privileged username to whom the mailbox should be shared to. This
>>> results in a invalid configuration. Users get confused, since they
>>> passed on a valid email address in their ACL setup.
>>>
>>> /usr/local/www/default/mail/test@mydomain.localdomain/maildir/.Spam/dovecot-acl
>>>  
>>>
>>>
>>> user=leander@mydomain.localdomain eilrwts
>>> ^^ works
>>>
>>> /usr/local/www/default/mail/leander@mydomain.localdomain/maildir/dovecot-acl
>>>  
>>>
>>>
>>> user=test@mydomain.localdomain eilrwts
>>> ^^ works
>>>
>>> /usr/local/www/default/mail/test@mydomain.localdomain/maildir/.Drafts/dovecot-acl
>>>  
>>>
>>>
>>> user=Leander@MyDomain.LocalDomain eilrwts
>>> ^^ Doesn't work
>>>
>>> Best regards
>>> Leander Schäfer
>> Hi! Did you know you can use %Lu instead of %u to force lowercasing?
>>
>> Aki

Re: NFSv4 and Maildir

2016-09-30 Thread Noel Butler 

On 01/10/2016 08:27, Joseph Tam wrote:

we have a setup with (CentOS 6) Director+Dovecot, Maildir as storage 
on

NetApp NFS v3. Every time I try to switch to NFS v4 I found issue with
lock (and others). So for me NFSv4 with Maildir is "unstable" or need 
a

fine tuning that I don't know.


I found the same thing, and turning off write delegation seemed
to have solved the problem.  I still don't know why, though.

Joseph Tam 


write delegation is disabled by default on NetApp with v4, or have they 
changed this now?

0x7FD036C7.asc
Description: application/pgp-keys

Re: Bug: Shared Mailbox - Case Sensitivity

2016-09-30 Thread Leander Schäfer 

Am I missing something, or might this be a bug as it seems to me?


Am 16.09.16 um 14:21 schrieb Leander Schäfer:

Hi Aki,


Thanks for your advice. Yes, I'm aware of this. Yet lowercasing should 
be the default since Dovecot 2.1.x., isn't it? Yet I wouldn't know 
where exactly to implement this %L, since the ACLs are set through 
IMAP commands through the users mailclient like Thunderbird. So in 
other words, the email address to whom the user want to grant ACLs 
provided by the user's mailclient, has nothing to do with my auth 
backend where e.g. %u => %Lu would apply. PLease correct me if I'm 
wrong here.



It clearly looks like a bug of the internal processing of the 
"dovecot-acl-list" files. It simply lacks on a lowercase enforcement 
in the code, like it already seems to do for the "dovecot-acl" file.



Best regards

Leander Schäfer



Am 16.09.16 um 12:53 schrieb Aki Tuomi:


On 16.09.2016 12:54, Leander Schäfer wrote:

Hi,

unfortunately I found a bug in Dovecot's ACL handling for shared
mailboxes. It turns out Dovecot doesn't enforce lower casing the
privileged username to whom the mailbox should be shared to. This
results in a invalid configuration. Users get confused, since they
passed on a valid email address in their ACL setup.

/usr/local/www/default/mail/test@mydomain.localdomain/maildir/.Spam/dovecot-acl 



user=leander@mydomain.localdomain eilrwts
^^ works

/usr/local/www/default/mail/leander@mydomain.localdomain/maildir/dovecot-acl 



user=test@mydomain.localdomain eilrwts
^^ works

/usr/local/www/default/mail/test@mydomain.localdomain/maildir/.Drafts/dovecot-acl 



user=Leander@MyDomain.LocalDomain eilrwts
^^ Doesn't work

Best regards
Leander Schäfer

Hi! Did you know you can use %Lu instead of %u to force lowercasing?

Aki

Re: acl_group not working not working correctly

2016-09-30 Thread Leander Schäfer 

Any idea?

Am 17.09.16 um 00:44 schrieb Leander Schäfer:

Hi,

I'm trying to setup group based ACLs coming from OpenLDAP. My setup 
doesn't require a POSIX Group match. In the Dovecot configuration file 
I have this: "user_attrs = [...], mailAclGroups=acl_groups" as well as 
"acl = vfile:/usr/local/etc/dovecot/global-acls:cache_secs=300". The 
user has "public" in the LDAP attribute "mailAclGroups". It seems to 
get everything right. I checked with doveadm - and I see public ist 
listed as expected:


cat /var/log/debug.log
[...]
Sep 16 23:39:04 WM-01 dovecot: auth: Debug: client passdb out: 
OK   1   user=leander@mydomain.localdomain acl_groups=public

[...]

cat /usr/local/etc/dovecot/global-acls
INBOX owner lrwstipekxa
Drafts owner lrwstipeka
Sent owner lrwstipeka
Spam owner lrwstipeka
Trash owner lrwstipeka
Public authenticated l
Public group-override=public lrwstipekx
Public/* group-override=public lrwstipekx


doveadm mailbox list -u leander@mydomain.localdomain
Drafts
Sent
Trash
Spam
Shared
Public
Public/Service Center
Shared/test@mydomain.localdomain
Shared/test@mydomain.localdomain/Drafts
Shared/test@mydomain.localdomain/Sent
Shared/test@mydomain.localdomain/Trash
Shared/test@mydomain.localdomain/Spam
INBOX


But here comes the strange thing: telnet equal to Thunderbird:
. LIST "" "*"
* LIST (\HasNoChildren \Drafts) "/" Drafts
* LIST (\HasNoChildren \Sent) "/" Sent
* LIST (\HasNoChildren \Trash) "/" Trash
* LIST (\HasNoChildren \Junk) "/" Spam
* LIST (\Noselect \HasChildren) "/" Shared
* LIST (\HasChildren) "/" Shared/test@mydomain.localdomain
* LIST (\HasNoChildren) "/" Shared/test@mydomain.localdomain/Drafts
* LIST (\HasNoChildren) "/" Shared/test@mydomain.localdomain/Sent
* LIST (\HasNoChildren) "/" Shared/test@mydomain.localdomain/Trash
* LIST (\HasNoChildren) "/" Shared/test@mydomain.localdomain/Spam
* LIST (\HasNoChildren) "/" INBOX
. OK List completed (0.000 + 0.000 + 0.092 secs).


Public and Public/* shoul be listed as well, but it isn't. Any idea 
why it is behaving like this?

Thanks

Best regards
Leander Schäfer

Re: NFSv4 and Maildir

2016-09-30 Thread Joseph Tam 

we have a setup with (CentOS 6) Director+Dovecot, Maildir as storage on
NetApp NFS v3. Every time I try to switch to NFS v4 I found issue with
lock (and others). So for me NFSv4 with Maildir is "unstable" or need a
fine tuning that I don't know.


I found the same thing, and turning off write delegation seemed
to have solved the problem.  I still don't know why, though.

Joseph Tam

Shared folder in a sharded cluster setup

2016-09-30 Thread Peer Heinlein 


Hi!

With Dovecot Director and Proxy or the new (great!) TAG-feature from
Dovecot it's easy to set up a shared IMAP-Cluster with individual local
filesystems.

But I'm unsure if it's possible to build a setup where shared mailboxes
still can work.

If user A is on Cluster (1) and user B is on (2),
and Cluster (1) does not have access to the mail-home from B on (2),

then user A can not reach the shared folders provided from User B on (2).

I hope that there is a kind of backend-proxy-mechanism, so that the imap
process of A on (1) can imap-proxy the requests for the shared folder to
a node from cluster shard (2).

And: To be exact, the imap process on (1) should forward the request to
cluster (2) by the director system to make sure, that the connection
will terminate on the right active backend of User B.


This sounds like a special problem if local filesystems with mdbox are
used and I now the great features of using Dovecot on Object Store,
where every node can check out all mail-locations from all users.

But especially on obox-systems it is very important that requests for a
user are always terminated on the same backend. So how can shared
folders work there?! Node (1) can not checkout the shared folders from
User B if his obox storage is already active on another host (2)!

Peer



-- 
Heinlein Support GmbH
Schwedter Str. 8/9b, 10119 Berlin

http://www.heinlein-support.de

Tel: 030 / 405051-42
Fax: 030 / 405051-19

Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht
Berlin-Charlottenburg,
Geschäftsführer: Peer Heinlein -- Sitz: Berlin

Maildir Expunged GUID mismatch for UID

2016-09-30 Thread Steven Xu 
 

Dovecot version:2.2.25

Since we used to keep our email files on widows server, I made the following
changes   in  maildir-storage.h

#define MAILDIR_EXTRA_SEP ','

#define MAILDIR_INFO_SEP_S ":" to "+".

 

Everything seems working except EXPUNG,  The dovecot log is flooded by
messages like following:

imap(x): Error: Mailbox INBOX: Expunged GUID mismatch for UID 7039

 

 

Then I read the source code, and found the following lines in
maildir-sync-index.c

 

T_BEGIN {

   guid = maildir_uidlist_lookup_ext(ctx->mbox->uidlist, uid,

MAILDIR_UIDLIST_REC_EXT_GUID);

   if (guid == NULL)

 guid = t_strcut(filename, ':');

   mail_generate_guid_128_hash(guid, guid_128);

  } T_END;

 

I have to change the code to guid = t_strcut(filename, '+');

 

 

So,  should MAILDIR_EXTRA_SEP be used here instead of  ':'?   

 

Thanks,

 

Steven

Re: doveadm backup fails (compromised single attachment storage)

2016-09-30 Thread Webert de Souza Lima 
by SAS I meant SIAS (Single Instance Attachment Storage).

On Thu, Sep 29, 2016 at 9:33 AM Webert de Souza Lima 
wrote:

> Hi,
>
> A couple of months ago I had a problem with Single Attachment Storage
> after infrastructure migration;
>
> All mailboxes were rsynced to another filesystem, and that may have broken
> Single Attachment Storage. Many, many (if not all) mailboxes show the below
> logs on dovecot:
>
> imap(f...@bar.com): Error:
> read(attachments-connector(zlib(/dovecotdir/mail/
> bar.com/foo/mailboxes/INBOX/dbox-Mails/u.26426))) failed:
> read(/dovecotdir/attach/
> bar.com/de/86/de8673894d6fb3f4460e3c26436eefa9a73517fa0f000452f553822367220761502e1d0ce220eee5aa9acf232df0adebf40cce90b57d2e60e1eb9c9ef21671fa-b0d3411772c1495753619331bd36-43cea6154b3275573b089331bd36-26426[base64:19
> 
> b/l]) failed: open(/dovecotdir/attach/
> bar.com/de/86/de8673894d6fb3f4460e3c26436eefa9a73517fa0f000452f553822367220761502e1d0ce220eee5aa9acf232df0adebf40cce90b57d2e60e1eb9c9ef21671fa-b0d3411772c1495753619331bd36-43cea6154b3275573b089331bd36-26426)
> failed: No such file or directory
>
>
> When that happens, the MUA keeps syncing forever.
>
> Now, I need to migrate all mailboxes (again) to another dovecot instance
> (with no SAS), which works perfectly for new users but when I try to
> migrate users from my current dovecot server for this new server, I get
> such errors again, and I can't migrate:
>
> 2016-09-29T12:20:50.995934059Z Sep 29 12:20:50 dsync-server(f...@bar.com):
> Error: dsync(cf7d091311eb):
> read(attachments-connector(zlib(/dovecotdir/mdbox/bar.com/foo/storage/m.1)))
> failed: read(/dovecotdir/attach/
> bar.com/0c/df/0cdf86b1920938fe3a043f87e2ee9e63dda276bd5b9fba687e4a0c63d181c3b6ebdb96a9517f048c963db71404ad5d14e896e2e67b7abb0c9e107aed5c15ecf1-430ea904dff46757ba179331bd36[base64:18
> 
> b/l]) failed: open(/dovecotdir/attach/
> bar.com/0c/df/0cdf86b1920938fe3a043f87e2ee9e63dda276bd5b9fba687e4a0c63d181c3b6ebdb96a9517f048c963db71404ad5d14e896e2e67b7abb0c9e107aed5c15ecf1-430ea904dff46757ba179331bd36)
> failed: No such file or directory (last sent=mail, last recv=mail_request
> (EOL))
>
> Is there a way to fix the attachments problem? (I know I can't recover
> such files, that's Ok)
> Is there a way to migrate (dsync backup) ignoring such problems?
>
> Thanks in advance.
>

Re: [patch] Improved error checking for the dovecot-antispam-plugin

2016-09-30 Thread Robert Munteanu 
Hi,

Has this slipped off the radar or is it somehow not suitable for inclusion?

Thanks,

Robert

On Thu, Aug 18, 2016 at 6:16 PM, Robert Munteanu
 wrote:
> (snip)
>
>> I have no issue in resending a new version of the patch with better
>> error reporting, will do so in the following days.
>>
>> Robert
>
> I've attached a second version of the patch, feel free to consider any
> of them for inclusion.
>
> Thanks,
>
> Robert
>
>
> --
> http://robert.muntea.nu/



-- 
http://robert.muntea.nu/

Re: NFSv4 and Maildir

2016-09-30 Thread Alessio Cecchi 


Il 23/09/2016 14:31, Robert Blayzor ha scritto:

Recently moving to newer storage platforms for mailbox storage so looking at 
moving mounts from NFSv3 with lots of issues with locking and caching to NFSv4.

There seems to be a lot of benefits to v4 along with some other new features, 
namely “delegation”.

So the question boils down to, to delegate or not delegate on Maildir storage. 
There may be many reasons based on actual platform why to do (or not to do 
this), but I want to get the general opinion from others that may have more 
experience with this. Our setup is several FreeBSD 10.x clients running 
Dovecot/Exim, NetApp NFS mail storage (probably moving to TrueNAS) and using F5 
load balancers for client side connections/SSL offload.

From what I’ve found (and what i’ve read in the RFC) is that delegation seems 
to work best when there is NOT a lot of file contention from clients accessing 
the same files. I realize that in some situations many people are using 
director to try and keep users on the same client; in our case we’re doing it 
with F5 iRules. The F5 iRules work great for POP3 and IMAP session persistence, 
but unfortunately that doesn’t work for SMTP and Dovecot LDA, so we still have 
possible race conditions from the MTA’s delivering into “INBOX”. (mostly 
dovecot indexes updating at the same time).

So the big question is, who is using Dovecot with maildirs with NFSv4 mounts. 
What has your experience been? Are you using delegation?  By choice and why did 
you come to that decision.

I’m drawing up the conclusion that if you can *mostly* control client control 
to specific files (ie: directing access to a mailbox to come from one client), 
then delegation might be ok. However, if you’re not using director and have 
several NFS mail clients racing to access mailboxes, then delegation might turn 
into chaos.


Your comments welcome and appreciated.


Hi Robert,

we have a setup with (CentOS 6) Director+Dovecot, Maildir as storage on 
NetApp NFS v3. Every time I try to switch to NFS v4 I found issue with 
lock (and others). So for me NFSv4 with Maildir is "unstable" or need a 
fine tuning that I don't know.


--
Alessio Cecchi
Postmaster @ http://www.qboxmail.it
https://www.linkedin.com/in/alessice