[Dovecot] Quota Problems with LMTP in HG 62a930eb22b5
Since updating to the latest HG these errors occur. Nothing else changed in the
config:
$ dovecot --version
2.2.beta1 (62a930eb22b5)
== /var/log/dovecot/dovecot.log ==
Feb 18 09:47:32 spectre dovecot: lmtp(14340): Connect from local
Feb 18 09:47:32 spectre dovecot: lmtp(14340, t...@leuxner.net): Error:
mkdir_parents(/var/vmail/domains/leuxner.net/tlx/mdbox) failed: File exists
Feb 18 09:47:32 spectre dovecot: lmtp(14340, t...@leuxner.net): Error:
mkdir_parents(/var/vmail/domains/leuxner.net/tlx/mdbox) failed: File exists
Feb 18 09:47:32 spectre dovecot: lmtp(14340, t...@leuxner.net): Error: dict
quota: Quota update failed, it's now desynced
$ doveconf -n
# 2.2.beta1 (62a930eb22b5): /etc/dovecot/dovecot.conf
# OS: Linux 3.2.0-0.bpo.4-amd64 x86_64 Debian 6.0.6
auth_cache_size = 16 k
auth_verbose = yes
mail_location = mdbox:~/mdbox
mail_plugins = acl quota stats
mailbox_list_index = yes
namespace {
list = yes
location = mdbox:/var/vmail/public:INDEXPVT=~/mdbox/public
prefix = Public/
separator = /
subscriptions = no
type = public
}
namespace inbox {
hidden = no
inbox = yes
location =
mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox Sent {
special_use = \Sent
}
mailbox Sent Messages {
special_use = \Sent
}
mailbox Trash {
special_use = \Trash
}
prefix =
separator = /
type = private
}
passdb {
args = username_format=%u /var/vmail/auth.d/%d/passwd
driver = passwd-file
}
plugin {
acl = vfile:/var/vmail/conf.d/%d/acls:cache_secs=300
mail_log_events = expunge mailbox_delete
quota = dict:user::file:%h/mdbox/dovecot-quota
quota_rule = *:storage=1GB
quota_rule2 = Trash:storage=+10%%
sieve = ~/.dovecot.sieve
sieve_dir = ~/sieve
sieve_global_dir = /var/vmail/conf.d/%d/sieve
stats_refresh = 30s
stats_track_cmds = yes
}
protocols = imap lmtp
quota_full_tempfail = yes
service auth-worker {
unix_listener auth-worker {
user = doveauth
}
user = doveauth
}
service auth {
unix_listener /var/spool/postfix/private/auth {
group = postfix
mode = 0660
user = postfix
}
user = doveauth
}
service imap-login {
inet_listener imap {
address = 1.2.3.4
port = 143
}
inet_listener imaps {
port = 0
}
}
service lmtp {
unix_listener /var/spool/postfix/private/dovecot-lmtp {
group = postfix
mode = 0660
user = postfix
}
}
service stats {
fifo_listener stats-mail {
mode = 0600
user = vmail
}
}
ssl_ca = /etc/ssl/certs/SSL123_CA_Bundle.pem
ssl_cert = /etc/ssl/certs/host_domain_tld.crt
ssl_key = /etc/ssl/private/host_domain_tld.key
syslog_facility = local1
userdb {
args = username_format=%u /var/vmail/auth.d/%d/passwd
driver = passwd-file
}
verbose_proctitle = yes
protocol lmtp {
mail_plugins = acl quota stats sieve
}
protocol imap {
mail_max_userip_connections = 20
mail_plugins = acl quota stats mail_log notify imap_acl imap_quota imap_stats
}
signature.asc
Description: Digital signature
Re: [Dovecot] Defer emails until the user is ready to deal with them (snooze button / bring-forward file / bring-up file for emails)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Sat, 16 Feb 2013, Chris Laif wrote: Thanks Timo, at least one person understands me ;-) Unfortunately, both solutions require programming skills far better than I have. So I start enjoying my weekend doing other things ... thank you for your great work on dovecot! I like Daniel's idea: + Have Thunderbird move unwanted messages to Delayed/time + Have a cron job on the server that moves the messages back to INBOX. - -- Steffen Kaiser -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) iQEVAwUBUSHua13r2wJMiz2NAQK7NAf+MEujHadBqNN7+7EWphjQj31vuEq/jFW7 xNY+jE9s/IBd0hmUY1cQLS6/X4n4EOOy4lAhzJIkE4/x8pqlQhlaA7L5kR8zkcK0 pNOzJHqOoWq0NL83WUf8kq/2BP6HUqL0nWX0Dgz5sWYvvXlGmKHND3eNZ5WX9IME Yj8QEbCDTqnMAh/jGjJVW8tzHtoGVBpGQ8EJbjUh8/QIciWtEVvz8ofmgz/5vpL3 r5XKiNHMyZs9dhaLvMsos7Qhln89XvANir8GAE0CFEFm+8mJDzzQU3RkAkj/ohT2 9MSYCQGDCiS3hcazZTRiCNvIup+t4vXGICVCtHjZHr8zUwsNqp5PCw== =070+ -END PGP SIGNATURE-
Re: [Dovecot] Dovecot-2.1.14 - pop3 processes always hangs forever - another follow-up
Le 18 févr. 2013 à 06:21, Timo Sirainen t...@iki.fi a écrit :
On Sun, 2013-02-17 at 12:24 +0100, Axel Luttgens wrote:
Le 17 févr. 2013 à 05:47, Timo Sirainen a écrit :
[...]
The gethostbyname() call is used to figure out the current system's domain.
Why it would be hanging there, I don't really know. That call anyway isn't
fully required, you could just replace it with hent = NULL;
Yes, looks like those gethostbyname() require some investigations here...
I'll have a look and provide my findings here.
Very modest findings; anyway, here they are, should they be of some help for
nailing the causes down.
First, having the db query returning only a virtual uid (eg, 12), thus
making use of a system group set with mail_gid (e.g., 999), those hangs do not
occur anymore.
On the other hand, this quick one:
#include syslog.h
#include netdb.h
#define NULL ((void *)0)
int main( int argc, const char * argv[])
{
struct hostent *hent;
hent = gethostbyname(ALMba.local);
if ( hent != NULL )
{
syslog(LOG_NOTICE|LOG_MAIL, name returned by
gethostbyname(): %s\n, hent-h_name);
}
return(0);
}
compiled as ghbn and with perms set to:
-rwsr-sr-x 1 12 12 8816 18 fév 09:08 ghbn
doesn't hang and yields:
[ASLMessageID 385162] [Time 1361175967] [TimeNanoSec 451292000] [Level
5] [PID 5616] [UID 12] [GID 12] [ReadGID 80] [Host ALMba.local] [Sender
ghbn] [Facility mail] [Message name returned by gethostbyname(): almba.local]
It could thus be inferred that the problems encountered here are related to
virtual gids only when setting up child processes from within Dovecot.
Don't know whether this impacts those calls to gethostbyname() only, or if
other side-effects may be expected as well.
Now, would all this be a Mac OS X thing only?
Or are such problems liable to occur on other platforms as well?
In the meantime, I don't really like the idea of messing with a general
purpose function, my_hostdomain(), just to avoid a local problem...
Wouldn't you have a better idea? ;-)
Solution for v2.2: http://hg.dovecot.org/dovecot-2.2/rev/a6b40687c0a4
Wow, that's a good idea. :-)
This should indeed circumvent those hangs, but also avoid many rather costly
system calls.
It seems it could be easily back ported to 2.1.x.
I'll try and report back here.
Many thanks,
Axel
Re: [Dovecot] Defer emails until the user is ready to deal with them (snooze button / bring-forward file / bring-up file for emails)
Am 18.02.2013 10:03, schrieb Steffen Kaiser: Have a cron job on the server that moves the messages back to INBOX. Hi Steffen, please describe how such cron should be invoked , created, edited, deleted with some user friendly interface i.e in some sieve webmail client or thunderbird, using tons of different guis for different jobs is not very nice, ideally such functions should be able to handle from a thunderbird plugin then etc this is not a general critic at this idea, but it must match real world sceanrios like isp setup etc at last Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich
Re: [Dovecot] Quota Problems with LMTP in HG 62a930eb22b5
* Thomas Leuxner t...@leuxner.net 2013.02.18 09:58: Since updating to the latest HG these errors occur. Nothing else changed in the config: $ dovecot --version 2.2.beta1 (62a930eb22b5) == /var/log/dovecot/dovecot.log == Feb 18 09:47:32 spectre dovecot: lmtp(14340): Connect from local Feb 18 09:47:32 spectre dovecot: lmtp(14340, t...@leuxner.net): Error: mkdir_parents(/var/vmail/domains/leuxner.net/tlx/mdbox) failed: File exists Feb 18 09:47:32 spectre dovecot: lmtp(14340, t...@leuxner.net): Error: mkdir_parents(/var/vmail/domains/leuxner.net/tlx/mdbox) failed: File exists Feb 18 09:47:32 spectre dovecot: lmtp(14340, t...@leuxner.net): Error: dict quota: Quota update failed, it's now desynced Seems there's more to the latest changes as a mail that has to be saved temporarily also breaks LMTP: Feb 18 13:00:32 spectre postfix/qmgr[21702]: 3Z8hWT6jxszFV: from=subscripti...@lists.juno.co.uk, size=460096, nrcpt=1 (queue active) == /var/log/dovecot/dovecot.log == Feb 18 13:00:32 spectre dovecot: lmtp(15093): Connect from local Feb 18 13:00:33 spectre dovecot: lmtp(15093): Error: Temp file creation to /tmp/dovecot.lmtp.�/usr/lib/dovecot/lmtp.15093. failed: No such file or directory Feb 18 13:00:33 spectre dovecot: lmtp(15093): Disconnect from local: Temporary internal failure (in DATA) signature.asc Description: Digital signature
Re: [Dovecot] Quota Problems with LMTP in HG 62a930eb22b5
On Mon, 2013-02-18 at 13:24 +0100, Thomas Leuxner wrote: * Thomas Leuxner t...@leuxner.net 2013.02.18 09:58: Since updating to the latest HG these errors occur. Nothing else changed in the config: $ dovecot --version 2.2.beta1 (62a930eb22b5) == /var/log/dovecot/dovecot.log == Feb 18 09:47:32 spectre dovecot: lmtp(14340): Connect from local Feb 18 09:47:32 spectre dovecot: lmtp(14340, t...@leuxner.net): Error: mkdir_parents(/var/vmail/domains/leuxner.net/tlx/mdbox) failed: File exists Feb 18 09:47:32 spectre dovecot: lmtp(14340, t...@leuxner.net): Error: mkdir_parents(/var/vmail/domains/leuxner.net/tlx/mdbox) failed: File exists Feb 18 09:47:32 spectre dovecot: lmtp(14340, t...@leuxner.net): Error: dict quota: Quota update failed, it's now desynced Not sure about these. Are they fixed now? Feb 18 13:00:33 spectre dovecot: lmtp(15093): Error: Temp file creation to /tmp/dovecot.lmtp.�/usr/lib/dovecot/lmtp.15093. failed: No such file or directory This is definitely fixed. I was worried about the environment getting cleared when coding it, but it appeared to work and looks like I didn't test well enough.
[Dovecot] statistics on proxy ???
Hi at all,
could I have imap statistics on my dovecot proxy server ??
Here my config:
# 2.1.13: /etc/dovecot/dovecot.conf
...
mail_plugins = stats
...
plugin {
...
stats_refresh = 30 secs
stats_track_cmds = yes
}
...
service stats {
fifo_listener stats-mail {
mode = 0666
}
}
...
protocol imap {
...
mail_plugins = stats imap_stats
}
File '/var/run/dovecot/stats' is always 0 byte:
# ls -la /var/run/dovecot/stats
srw---. 1 root root 0 Feb 18 14:24 /var/run/dovecot/stats
and 'dovecotadm' say always:
# doveadm stats dump session
doveadm(root): Info: no statistics available
Is there something wrong in my config, or stats are not available on
proxy ???
Cheers,
Marco
Re: [Dovecot] Quota Problems with LMTP in HG 62a930eb22b5
* Timo Sirainen t...@iki.fi 2013.02.18 14:18: == /var/log/dovecot/dovecot.log == Feb 18 09:47:32 spectre dovecot: lmtp(14340): Connect from local Feb 18 09:47:32 spectre dovecot: lmtp(14340, t...@leuxner.net): Error: mkdir_parents(/var/vmail/domains/leuxner.net/tlx/mdbox) failed: File exists Feb 18 09:47:32 spectre dovecot: lmtp(14340, t...@leuxner.net): Error: mkdir_parents(/var/vmail/domains/leuxner.net/tlx/mdbox) failed: File exists Feb 18 09:47:32 spectre dovecot: lmtp(14340, t...@leuxner.net): Error: dict quota: Quota update failed, it's now desynced Not sure about these. Are they fixed now? Feb 18 13:00:33 spectre dovecot: lmtp(15093): Error: Temp file creation to /tmp/dovecot.lmtp.�/usr/lib/dovecot/lmtp.15093. failed: No such file or directory This is definitely fixed. I was worried about the environment getting cleared when coding it, but it appeared to work and looks like I didn't test well enough. Both seem to be fixed with 2.2.beta1 (3ca7e0eaaf4b). Thanks Timo. signature.asc Description: Digital signature
Re: [Dovecot] statistics on proxy ???
On Mon, 2013-02-18 at 14:33 +0100, Marco Giunta wrote: could I have imap statistics on my dovecot proxy server ?? Nope, sorry. Dovecot proxy is very dummy and can't provide any but the most basic statistics, like number of connections, which you can get another way.
[Dovecot] Full-text search
Hi all I discovered that the full-text search (fts) plugin can work without SQUAT/LUCENE/SOLR backend. I.e., Dovecot creates separate indexes for header search in files dovecot.index and dovecot.index.cache. Even, the search by headers is fast enough, and can search for phrases. Also, it seems that this buillt-in search is faster than Solr-based search. But if I enable the Solr backend (fts_solr), then the FTS generic plugin built-in search becomes disabled. But, Solr-based Full-text search is faster if search inside message bodies. Also, it appears that the built-in search inside message bodies works too, but very slow (it seems that is because it is dumb file-based search and does not use indexes at all) So, my question is: is it possible to combine the built-in search with Solr or Lucene plugin-based search so that the first one searches by headers, and the second one works by bodies? This could make the advantages of both search methods combined. WBR, valery
Re: [Dovecot] Defer emails until the user is ready to deal with them (snooze button / bring-forward file / bring-up file for emails)
Robert Schetterer wrote: Am 18.02.2013 10:03, schrieb Steffen Kaiser: move the messages back to INBOX ideally such functions should be able to handle from a thunderbird plugin What about http://mailmindr.net/ Regards Daniel -- https://plus.google.com/103021802792276734820
Re: [Dovecot] dsync replication errors
On 18.02.2013, at 07:49, Timo Sirainen t...@iki.fi wrote: On Sun, 2013-02-17 at 12:30 +0200, Timo Sirainen wrote: (So yeah, ideally there should be checks for detecting hostname hash collisions..) Added to v2.2 hg: % doveconf -H dovecot%2d No duplicate host hashes in dovecot0 .. dovecot99 With doveconf -H dovecot%9d I do end in tons of reported collisions like ... | doveconf: Error: Duplicate host hashes: dovecot1368344 and dovecot2055005 | doveconf: Error: Duplicate host hashes: dovecot2042008 and dovecot2056918 | doveconf: Error: Duplicate host hashes: dovecot1844965 and dovecot2058312 (No wonder, I am running 2.1 replicator with identical local hostnames for some time now.) ... and ending with: | Killed doveconf -H without the template it attempts to detect it from the current hostname. mail doveconf -H doveconf: Fatal: Hostname 'xxx.yyy.tld' has no digits, can't verify JFTR and regards, Michael
Re: [Dovecot] dsync replication errors
On 18.02.2013, at 07:07, Timo Sirainen t...@iki.fi wrote: On 17.2.2013, at 22.04, Michael Grimm trash...@odo.in-berlin.de wrote: First of all: whenever you referred to hostname in this thread you have been using it as a synonym for the local part [1] of a FQDN, right? I mean what gethostname() function returns, which is what hostname command usually also returns. And yes, I think it's the local part always. I am not familiar with the gethostname() function within FreeBSD, but the hostname command normally returns your FQDN, if set. That has been the case because I didn't configure my service jails with FQDNs, thus a hostname couldn't return something else then the local hostname. Given that all my interpretations of your statements are correct I do have difficulties in understanding why a generic communication between Dovecot servers should be limited to enforcing different local parts of all Dovecot servers implied instead of different FQDN? That would make much more sense regarding uniqueness in hostnames, IMHO. Two servers like dovecot.forget-about.it and dovecot.you-name.it should be able to communicate generically, again: IMHO. I think systems named those would belong to different clusters and wouldn't need to communicate with each others. Well, now I do understand my misunderstanding: I did consider replication between different clusters a generic communication between Dovecot servers, as well. I looked through the code. The hostname (without domain) are currently used for: * maildir filenames * temporary filenames * authentication challenge strings in some auth mechanisms * logging So I think the hostname uniqueness matters mainly when using a shared filesystem (e.g. NFS). So, I'm confident that I may stick to identical local hostnames regarding both servers of mine. Thanks and with kind regards, Michael
Re: [Dovecot] dsync replication errors
On 18.2.2013, at 23.50, Michael Grimm trash...@odo.in-berlin.de wrote: % doveconf -H dovecot%2d No duplicate host hashes in dovecot0 .. dovecot99 With doveconf -H dovecot%9d I do end in tons of reported collisions like ... | doveconf: Error: Duplicate host hashes: dovecot1368344 and dovecot2055005 | doveconf: Error: Duplicate host hashes: dovecot2042008 and dovecot2056918 | doveconf: Error: Duplicate host hashes: dovecot1844965 and dovecot2058312 Sure there are going to be hash collisions at some point, but I highly doubt you're going to create a million server Dovecot cluster. :)
Re: [Dovecot] Defer emails until the user is ready to deal with them (snooze button / bring-forward file / bring-up file for emails)
Am 18.02.2013 21:36, schrieb Daniel Parthey: Robert Schetterer wrote: Am 18.02.2013 10:03, schrieb Steffen Kaiser: move the messages back to INBOX ideally such functions should be able to handle from a thunderbird plugin What about http://mailmindr.net/ Regards Daniel Hi Daniel, simply try it , and look if it fits to your needs, but i guess there is no real relation to the imap server you use, it may be a client function ,only, please test and report Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich
Re: [Dovecot] Private message flags in shared mailboxes
On Sat, 2013-02-16 at 20:19 +0200, Timo Sirainen wrote: I'm wondering a bit how these should work. Comments welcome. Well, I guess: In v2.2 the recommended way to enable private flags for shared mailboxes is to create private indexes with :INDEXPVT=path. This creates dovecot.index.pvt[.log] files which only contain message UIDs and flags. The indexes are updated somewhat lazily by updating them whenever user has the shared mailbox opened and it gets synced. 1. What would be a good place to configure which flags are shared and which are private? Currently it forces \Seen flag to be private and others not. With Maildir I used to have the idea of configuring these in dovecot-shared file, but that's a rather annoying extra file. Then again it could be cached into dovecot.index. Another possibility would be in dovecot-acl file, but public mailboxes might be enabled without ACL plugin. Mailbox attributes (for URLAUTH and for future METADATA) also exist in v2.2, but they require configuring a dict for them, which isn't very efficient either to read every time a mailbox is accessed. Mailbox's admin user should be able to do this somehow via IMAP protocol also. METADATA would pretty much be required for that I guess. Probably won't happen before v2.3. 2. Private flags aren't currently inherited from the primary (shared) flags. With user-shared mailboxes this is good, because the owner doesn't have private indexes for the mailboxes, so when owner reads a mail it gets a \Seen flag in the primary index. If that mailbox was shared to someone else, the \Seen flag shouldn't be set. But this also means that for truly shared (public) mailboxes that have no owner you can't set initial flags with Sieve's addflag command (e.g. \Seen flag to high-scored mails in Spam). I guess it could be possible to change this so that the inheritance is done for public namespaces but not for shared namespaces, but is that really a good idea either?.. Initial private flags are copied now from shared flags for mailboxes in public namespaces, but not elsewhere. 3. Currently COPY/APPEND ignore any private flags (which also means dsync won't set them always). Fixing this would require remembering these flags, committing the save to primary index, syncing the private index to get the new mails added there, then updating their flags (or doing it directly at sync stage with some kludging code). To avoid race conditions the private index would need to be locked before committing the save. I guess this is doable.. Fixed. 4. Private keywords aren't supported. I guess not much worse than system flags, but more code complexity. Wonder if anyone would care about them. Private \Seen flag is good enough for now.
