Re: Additional userdb variables in passwd [was Re: Dovecot Replication - Architecture Endianness?]
On 05/07/2015 02:32 PM, Reuben Farrelly wrote:
On 7/05/2015 7:49 AM, Timo Sirainen wrote:
On 06 May 2015, at 13:52, Reuben Farrelly reuben-dove...@reub.net wrote:
On 4/05/2015 11:06 PM, Teemu Huovila wrote:
Also is there a way to restrict replication users aside from a crude hack
around system first and last UIDs?
You can set the userdb to return an empty mail_replica variable for users
you want to exclude from replication.
http://hg.dovecot.org/dovecot-2.2/rev/c1c67bdc8752
br,
Teemu Huovila
One last question. Is it possible to achieve this with system users and
PAM or do I need to basically create a new static
userdb for system users?
You can create a new userdb passwd-file that adds extra fields. So something
like:
userdb {
driver = passwd
result_success = continue-ok
}
userdb {
driver = passwd-file
args = /etc/dovecot/passwd.extra
skip = notfound
}
This doesn't seem to work for me and my config has that exact config. My
password.extra file has just one line for the one
account I am testing with at the moment:
user1:::userdb_mail_replica=tcps:lightning.reub.net:4813,userdb_mail_replica=tcp:pi.x.y:4814
This breaks access for other system users such as my own account which do not
have entries:
ay 7 21:19:06 tornado.reub.net dovecot: imap-login: Internal login failure
(pid=22573 id=1) (internal failure, 1 successful
auths): user=reuben, auth-method=PLAIN, remote=2001:44b8:31d4:1311::50,
local=2001:44b8:31d4:1310::20, TLS
which then starts soon spitting this out 10s of times per second in the mail
log:
May 7 21:19:32 tornado.reub.net dovecot: auth-worker(23738): Error: Auth
worker sees different passdbs/userdbs than auth
server. Maybe config just changed and this goes away automatically?
This is with -hg latest as of now.
This system uses PAM for local users. Do I need to replicate all of the
system users including those who do not need any extra
settings, in the passwd.extra file too?
Is my syntax above for two mail_replica servers correct?
A bit unsure about the config syntax, so I can not advice on that, but there
were some bugs in auth yesterday. Maybe you could
retest with f2a8e1793718 or newer. Make sure configs on both sides are in sync.
Thank you for your continued testing,
Teemu Huovila
Different mdbox_rotate_size for primary and alternate storage
Hello. In order to speed up backups of very very old messages I would like to set two different limits for mdbox_rotate_size. Like, 50M for primary storage and 100M or larger for alternate storage. There is no mention in docs or such a possibility, so I assume it is not possible. Is that correct? While I am at it, is it possible to configure primary storage as maildir (sturdy indexes) and altstorage as mdbox (more delicate indexes)? Thanks, Paolo
Re: PublicFolders using Maildir and INDEXPVT in Dovecot v2.2
Do we have a way to keep the user \Seen flags in public folders
when an email is moved in another folder?
Sample test to reproduce:
Step a - UserA and UserB have both read the email in PublicFolderA.
Step b - UserA moves the email to PublicFolderB. UserA still sees the
email as read (with the \Seen flag), this is expected behaviour and we
are ok with this.
Step c - UserB sees the email in PublicFolderB as an unread message!? He
is puzzled since he has already read this message and asks why this is
happening. Can we correct this behaviour?
Thank you all,
Panos.
On 5/5/2015 01:17, Panayiotis Fafakos wrote:
Dear all,
we have succesfully configured Dovecot v2.2.13 in debian wheezy 7.8
(using backports)
using Maildir structure, to use private index files for the \Seen flag
on a per user basis.
All users access their emails and Public Folders using IMAP protocol.
The problem is that when a user moves an email from publicFolderA to
publicFolderB under the same namespace
the other users see this message as unread, although they have
actually read it when it was in publicFolderA.
Please note that this is an old message which has been moved , it was
not copied, so the actual UID should be the same...
Is there a way to keep the \Seen flag for the messages that are moved
from folder to folder?
Is there a way to keep the \Seen flag in a database, so that we can
ignore the folder structure and only check the message UIDs? We could
use MySQL, PgSQL or even SQLite...
Below follows the Public-Folder namespace declaration:
--
namespace {
inbox = no
location =
maildir:/var/vmail/Public-Folders:LAYOUT=fs:INDEXPVT=~/Maildir/public/%u
prefix = Public-Folders/
separator = /
subscriptions = no
type = public
}
--
With the above system configuration we have the complete folder
structure under ~/Maildir/public/%u, and many log files, one for each
folder a user has accessed.
Could we only have one index file for each user for all the public
folder structure under the same namespace?
Kind regards to all,
Panayiotis Fafakos
Re: PublicFolders using Maildir and INDEXPVT in Dovecot v2.2
Dear Timo, thank you very much for your answer and for the wonderful DOVECOT project. Can you please tell us where is the code that moves the index record for the specified email, so that we can perhaps provide a manually updated list of users, that we want dovecot to update also. This would be a custom code modification which we would need to keep-up-to-date, and we could also publish it if anyone would be interested. We are supporting companies up to 10 users, so this minor change would not be a problem to maintain. i.e. we could have a list of users in the main departmental public folder who's index would also be updated once an email in the underlining folder structure changes folder. We are also a software company and would be very interested in investing to get to know more for the dovecot project. Thank you in advance for your support, Panos. On 8/5/2015 18:42, Timo Sirainen wrote: On 08 May 2015, at 18:24, Panayiotis Fafakos p...@wisdomsoftware.net wrote: Do we have a way to keep the user \Seen flags in public folders when an email is moved in another folder? Sample test to reproduce: Step a - UserA and UserB have both read the email in PublicFolderA. Step b - UserA moves the email to PublicFolderB. UserA still sees the email as read (with the \Seen flag), this is expected behaviour and we are ok with this. Step c - UserB sees the email in PublicFolderB as an unread message!? He is puzzled since he has already read this message and asks why this is happening. Can we correct this behaviour? No, there's no way to fix this without major changes to how Dovecot works. Per-user seen flags are stored in private per-user index files. When user A moves mail to another folder the shared index and user A's index are updated to copy the flags. User A doesn't know what other users might have the folder accessible (and especially both source and destination folder). Even if it did know, now moving a mail might involve updating a lot of indexes every time a mail is moved, which is way too slow. One possible future solution would be to move more towards GMail-like labels instead of folders. We have beginnings of such code. I'm not sure yet how that could be made to work with shared folders.
Re: PublicFolders using Maildir and INDEXPVT in Dovecot v2.2
On 08 May 2015, at 18:24, Panayiotis Fafakos p...@wisdomsoftware.net wrote: Do we have a way to keep the user \Seen flags in public folders when an email is moved in another folder? Sample test to reproduce: Step a - UserA and UserB have both read the email in PublicFolderA. Step b - UserA moves the email to PublicFolderB. UserA still sees the email as read (with the \Seen flag), this is expected behaviour and we are ok with this. Step c - UserB sees the email in PublicFolderB as an unread message!? He is puzzled since he has already read this message and asks why this is happening. Can we correct this behaviour? No, there's no way to fix this without major changes to how Dovecot works. Per-user seen flags are stored in private per-user index files. When user A moves mail to another folder the shared index and user A's index are updated to copy the flags. User A doesn't know what other users might have the folder accessible (and especially both source and destination folder). Even if it did know, now moving a mail might involve updating a lot of indexes every time a mail is moved, which is way too slow. One possible future solution would be to move more towards GMail-like labels instead of folders. We have beginnings of such code. I'm not sure yet how that could be made to work with shared folders.
Cant use doveadm to set ACL . [request for help]
Hi, i keep getting error for using doveadm acl get commands. Below is the output for grep 'socket_path' :- grep 'socket_path' /etc/dovecot/dovecot.conf auth_socket_path = /var/run/dovecot/auth-master auth_socket_path = /var/run/dovecot/auth-master [root@mail root]# doveadm acl get -u b...@mydomain.net -S /var/run/dovecot/auth-master -m Inbox doveadm(b...@mydomain.net): Error: doveadm server sent invalid handshake: VERSION11 doveadm(b...@mydomain.net): Error: /var/run/dovecot/auth-master: Internal failure for b...@mydomain.net ID Global Rights
Full text search indexes not used for header/body OR queries?
I've noticed that when using Lucene full text search, most queries use the indexes and/or header cache and are fast: . SEARCH BODY test . OK Search completed (0.001 secs). . SEARCH SUBJECT test . OK Search completed (0.053 secs). . SEARCH BODY test SUBJECT test . OK Search completed (0.002 secs). . SEARCH OR SUBJECT test FROM test . OK Search completed (0.093 secs). But an OR query that mixes headers and body does not use the available FTS indexes for the BODY part and is slow: . SEARCH OR BODY test SUBJECT test * OK Searched 62% of the mailbox, ETA 0:05 * OK Searched 70% of the mailbox, ETA 0:04 . OK Search completed (15.147 secs). Is this the expected behavior? Since the FTS code can handle an AND of header and body searches, I'm surprised it doesn't do the same for an OR. I noticed this while tracking down poor performance in Thunderbird, which issues searches like this: UID SEARCH RETURN (ALL) (OR FROM Evelyn (OR SUBJECT Evelyn (OR TO Evelyn (OR CC Evelyn BODY Evelyn NOT DELETED These are slow even with FTS enabled because of this behavior. I'm using Dovecot 2.1.7 from Debian wheezy. (I know this is outdated; however, I've examined the 2.1.x and 2.2.x changelogs and found no mention of it.) -- Robert L Mathews, Tiger Technologies, http://www.tigertech.net/
Re: Additional userdb variables in passwd [was Re: Dovecot Replication - Architecture Endianness?]
On 8/05/2015 6:10 PM, Teemu Huovila wrote:
On 05/07/2015 02:32 PM, Reuben Farrelly wrote:
On 7/05/2015 7:49 AM, Timo Sirainen wrote:
On 06 May 2015, at 13:52, Reuben Farrelly
reuben-dove...@reub.net wrote:
On 4/05/2015 11:06 PM, Teemu Huovila wrote:
Also is there a way to restrict replication users aside
from a crude hack around system first and last UIDs?
You can set the userdb to return an empty mail_replica
variable for users you want to exclude from replication.
http://hg.dovecot.org/dovecot-2.2/rev/c1c67bdc8752
br, Teemu Huovila
One last question. Is it possible to achieve this with system
users and PAM or do I need to basically create a new static
userdb for system users?
You can create a new userdb passwd-file that adds extra fields.
So something like:
userdb { driver = passwd result_success = continue-ok }
userdb { driver = passwd-file args = /etc/dovecot/passwd.extra
skip = notfound }
This doesn't seem to work for me and my config has that exact
config. My password.extra file has just one line for the one
account I am testing with at the moment:
user1:::userdb_mail_replica=tcps:lightning.reub.net:4813,userdb_mail_replica=tcp:pi.x.y:4814
This breaks access for other system users such as my own account which
do not have entries:
ay 7 21:19:06 tornado.reub.net dovecot: imap-login: Internal login
failure (pid=22573 id=1) (internal failure, 1 successful auths):
user=reuben, auth-method=PLAIN, remote=2001:44b8:31d4:1311::50,
local=2001:44b8:31d4:1310::20, TLS
which then starts soon spitting this out 10s of times per second in
the mail log:
May 7 21:19:32 tornado.reub.net dovecot: auth-worker(23738):
Error: Auth worker sees different passdbs/userdbs than auth server.
Maybe config just changed and this goes away automatically?
This is with -hg latest as of now.
This system uses PAM for local users. Do I need to replicate all
of the system users including those who do not need any extra
settings, in the passwd.extra file too?
Is my syntax above for two mail_replica servers correct?
A bit unsure about the config syntax, so I can not advice on that,
but there were some bugs in auth yesterday. Maybe you could retest
with f2a8e1793718 or newer. Make sure configs on both sides are in
sync.
Thank you for your continued testing, Teemu Huovila
With -hg as of now it's still not any better:
tornado log # dovecot --version
2.2.16 (f2a8e1793718+)
tornado log #
===
# System users (NSS, /etc/passwd, or similiar). In many systems nowadays
this
# uses Name Service Switch, which is configured in /etc/nsswitch.conf.
userdb {
# doc/wiki/AuthDatabase.Passwd.txt
driver = passwd
# [blocking=no]
#args =
# Override fields from passwd
#override_fields = home=/home/virtual/%u
result_success = continue-ok
}
# Add some extra fields such as replication..
userdb {
driver = passwd-file
args = /etc/dovecot/passwd.extra
skip = notfound
}
==
May 8 22:59:11 tornado.reub.net dovecot: imap: Error: Authenticated
user not found from userdb, auth lookup id=586547201 (client-pid=29035
client-id=1)
May 8 22:59:11 tornado.reub.net dovecot: imap-login: Internal login
failure (pid=29035 id=1) (internal failure, 1 successful auths):
user=reuben, auth-method=PLAIN, remote=2001:44b8:31d4:1311::50,
local=2001:44b8:31d4:1310::20, TLS
It logs an awful lot of those lines in short succession also, at least
15 per second...
Reuben
Re: Full text search indexes not used for header/body OR queries?
As a followup to my own message: On 5/8/15 1:34 PM, Robert L Mathews wrote: I've noticed that when using Lucene full text search, most queries use the indexes and/or header cache and are fast [...] But an OR query that mixes headers and body does not use the available FTS indexes for the BODY part and is slow: This turned out to be my own fault because of a foolish mistake I made when testing. Dovecot actually works fine on all the search queries I mentioned, even in version 2.1.7. My apologies for the noise on the list. (My mistake was that when switching from Squat to Lucene, I didn't remove a local patch that prevented FTS from being used for header searches, because I thought the patch was only affecting Squat. That patch was to workaround what I reported in http://www.dovecot.org/list/dovecot/2014-May/096360.html. But the patch also affected Lucene.) -- Robert L Mathews, Tiger Technologies, http://www.tigertech.net/
