Re: Dbox and Exim
On 09/10/14 23:06, Timo Sirainen wrote: On 09 Oct 2014, at 15:20, C Peters chuck.pet...@gmail.com wrote: The Dovecot wiki [1] doesn't list either Exim, Postfix or anything else as supporting the dbox. I did some searching in the Exim archives and have found only one message mentioning dbox, sdbox or mdbox. [2] I think an RFC would go a long way towards getting the format supported. Has Timo, or anyone else, considered submitting an RFC to the IETF? No, and I don't want it to happen anytime in near future. The dbox format is highly Dovecot-specific, because it relies on Dovecot's index files. These index files are still changing and I'm not sure if they will ever be fully finished. I don't want any other software to even attempt to implement the same functionality. My question was how are we going to use it if Exim doesn't support it. As others said, use LMTP or dovecot-lda with Sieve filtering. Hi, I am sorry if I am presumptuous or out of topic, but it seems to me that the mail storage should be standardised. I am still using Maildir myself, simply because I like the simplicity, one file is one email, one directory is one email folder, for backup or restore. The only thing I don't like is the naming convention, but I don't know if a file system exists today to store such information in label or tags associated to each file. Regarding the optimisation, I think it's the file system's role, albeit I appreciate Dovecot's efforts to optimise the storage by taking in consideration file system's specificities. A feature I would like, is dovecot being able to store emails in databases, SQL or NoSQL. I would use this for archiving and queries purposes. MariaDB / Drizzle, or MongoDB, CouchDB, etc. There is plenty of choice. Maybe this storage scheme would be slower, but IMHO, this is not a problem when people just want to archive their emails on a remote computer. Maybe I could start this if I had some template or example to start. Kind regards, André.
2.2.14rc1 - dsync in backup mode still changes source permissions
Hi, It seems we are still able to reproduce this: http://www.dovecot.org/list/dovecot/2014-May/096367.html However... there's no longer any error-messages. It just silently changes permissions on some dovecot files in the source maildir. (most often dovecot-uidlist) We're running dsync as root, with hardwired userdb values for other reasons. So it has the OS permissions to change source. But still, running in backup shouldn't change source ever, should it? The command line is of this format - running on destination-host: # dsync -R -o mail_home=/users/user/maildir backup ssh -c arcfour -o StrictHostKeyChecking=no -i /root/.ssh/id-rsa-dsync source-host dsync -o mail_home=/users/user/maildir /Peter
Re: Dbox and Exim
Am 10.10.2014 um 09:53 schrieb Andre Rodier: On 09/10/14 23:06, Timo Sirainen wrote: On 09 Oct 2014, at 15:20, C Peters chuck.pet...@gmail.com wrote: The Dovecot wiki [1] doesn't list either Exim, Postfix or anything else as supporting the dbox. I did some searching in the Exim archives and have found only one message mentioning dbox, sdbox or mdbox. [2] I think an RFC would go a long way towards getting the format supported. Has Timo, or anyone else, considered submitting an RFC to the IETF? No, and I don't want it to happen anytime in near future. The dbox format is highly Dovecot-specific, because it relies on Dovecot's index files. These index files are still changing and I'm not sure if they will ever be fully finished. I don't want any other software to even attempt to implement the same functionality. My question was how are we going to use it if Exim doesn't support it. As others said, use LMTP or dovecot-lda with Sieve filtering. Hi, I am sorry if I am presumptuous or out of topic, but it seems to me that the mail storage should be standardised. Not really, its good to have many options I am still using Maildir myself, simply because I like the simplicity, one file is one email, one directory is one email folder, for backup or restore. The only thing I don't like is the naming convention, but I don't know if a file system exists today to store such information in label or tags associated to each file. Regarding the optimisation, I think it's the file system's role, albeit I appreciate Dovecot's efforts to optimise the storage by taking in consideration file system's specificities. A feature I would like, is dovecot being able to store emails in databases, SQL or NoSQL. I would use this for archiving and queries purposes. MariaDB / Drizzle, or MongoDB, CouchDB, etc. There is plenty of choice. agreed, its good to have more options Maybe this storage scheme would be slower, but IMHO, this is not a problem when people just want to archive their emails on a remote computer. why not using bcc stuff with i.e postfix perhaps see https://sys4.de/de/blog/2013/02/07/mailarchiv-mit-dovecot-und-postfix-sortiert-nach-datum-mailadressen-und-ein-ausgehend-unterordnern/ sorry german Maybe I could start this if I had some template or example to start. Kind regards, André. Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
Re: question: dovecot lda running as special user (vmail) or normal user (marcel)?
Dear Mailing List,
(version and dovecot -n at the bottom)
(Sorry for the bad English in this mail :) )
I'm new to dovecot and wanted to build my own mailserver using
dovecot+postfix.
I currently have a Problem with Permissions with my mailfolder.
I could solve it easily if i set it with chmod to 777. but thats no real
solution.
While reading my log files i see a lot messages like:
Oct 10 05:19:52 lda(owncloud): Error: user owncloud: Initialization failed:
Initializing mail storage from mail_location setting failed:
stat(/home/vmail/example.com/owncloud/mail) failed: Permission denied
(euid=100(owncloud) egid=1004(owncloud) missing +x perm: /home/vmail, dir
owned by 5000:5000 mode=0700)
Oct 10 05:19:52 lda(owncloud): Fatal: Invalid user settings. Refer to
server log for more information.
So Currently two Users need to acces the mail-folder:
1. The user itself (here: owncloud)
2. The vmail user
I want to use dovecot with virtual users. Now the question is: shouldn't
the directory be accessed only by the vmail user? and not by the owncloud
user?
Second Quest:
If it's correct that the access is made by 2 users: what rights do they
need?
I Cant add all users to the group vmail and set g+rwx Permissions (every
user could read mails from other users, and even edit them!)
Thanks a Lot.
Kind Regards,
Marcel
--
dovecot --version
2.2.13
dovecot -n
# 2.2.13: /etc/dovecot/dovecot.conf
# OS: Linux 3.13.0-37-generic x86_64 Ubuntu 14.04.1 LTS ext4
auth_mechanisms = plain login
auth_verbose = yes
info_log_path = /var/log/dovecot-info.log
log_path = /var/log/dovecot.log
mail_home = /home/vmail/example.com/%n
mail_location = maildir:/home/vmail/example.com/%n/mail:LAYOUT=fs
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags
copy include variables body enotify environment mailbox date ihave duplicate
namespace inbox {
inbox = yes
location =
mailbox Drafts {
auto = subscribe
special_use = \Drafts
}
mailbox Junk {
auto = subscribe
special_use = \Junk
}
mailbox Sent {
auto = subscribe
special_use = \Sent
}
mailbox Sent Messages {
auto = subscribe
special_use = \Sent
}
mailbox Trash {
auto = subscribe
special_use = \Trash
}
prefix =
}
passdb {
args = username_format=%u scheme=ssha512 /etc/dovecot/passwd.db
driver = passwd-file
}
plugin {
sieve = ~/.dovecot.sieve
sieve_after = /home/vmail/sieve-after
sieve_before = /home/vmail/sieve-before
sieve_dir = ~/sieve
}
protocols = imap sieve
service auth {
unix_listener /var/spool/postfix/private/dovecot-auth {
group = postfix
mode = 0660
user = postfix
}
}
ssl_cert = /etc/ssl/certs/ssl_main.crt
ssl_cipher_list =
ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AES:RSA+3DES:!ADH:!AECDH:!MD5:!DSS
ssl_client_ca_dir = /etc/ssl/certs
ssl_key = /etc/ssl/private/ssh_main_insecure
userdb {
args = uid=5000 gid=5000 home=/home/vmail/example.com/%n
driver = static
}
protocol imap {
imap_client_workarounds = delay-newmail tb-extra-mailbox-sep
mail_max_userip_connections = 10
}
protocol lda {
deliver_log_format = msgid=%m: %$
mail_plugins = sieve
postmaster_address = postmas...@example.com
quota_full_tempfail = yes
rejection_reason = Your message to %t was automatically rejected:%n%r
}
On Fri, Oct 10, 2014 at 10:13 AM, marcel.cochem
marcel.coc...@googlemail.com wrote:
Dear Mailing List,
(version and dovecot -n at the bottom)
(Sorry for the bad English in this mail :) )
I'm new to dovecot and wanted to build my own mailserver using
dovecot+postfix.
I currently have a Problem with Permissions with my mailfolder.
I could solve it easily if i set it with chmod to 777. but thats no real
solution.
While reading my log files i see a lot messages like:
Oct 10 05:19:52 lda(owncloud): Error: user owncloud: Initialization
failed: Initializing mail storage from mail_location setting failed:
stat(/home/vmail/example.com/owncloud/mail) failed: Permission denied
(euid=100(owncloud) egid=1004(owncloud) missing +x perm: /home/vmail, dir
owned by 5000:5000 mode=0700)
Oct 10 05:19:52 lda(owncloud): Fatal: Invalid user settings. Refer to
server log for more information.
So Currently two Users need to acces the mail-folder:
1. The user itself (here: owncloud)
2. The vmail user
I want to use dovecot with virtual users. Now the question is: shouldn't
the directory be accessed only by the vmail user? and not by the owncloud
user?
Second Quest:
If it's correct that the access is made by 2 users: what rights do they
need?
I Cant add all users to the group vmail and set g+rwx Permissions (every
user could read mails from other users, and even edit them!)
Thanks a Lot.
Kind Regards,
question: dovecot lda running as special user (vmail) or normal user (marcel)?
Dear Mailing List,
(version and dovecot -n at the bottom)
(Sorry for the bad English in this mail :) )
I'm new to dovecot and wanted to build my own mailserver using
dovecot+postfix.
I currently have a Problem with Permissions with my mailfolder.
I could solve it easily if i set it with chmod to 777. but thats no real
solution.
While reading my log files i see a lot messages like:
Oct 10 05:19:52 lda(owncloud): Error: user owncloud: Initialization failed:
Initializing mail storage from mail_location setting failed:
stat(/home/vmail/example.com/owncloud/mail) failed: Permission denied
(euid=100(owncloud) egid=1004(owncloud) missing +x perm: /home/vmail, dir
owned by 5000:5000 mode=0700)
Oct 10 05:19:52 lda(owncloud): Fatal: Invalid user settings. Refer to
server log for more information.
So Currently two Users need to acces the mail-folder:
1. The user itself (here: owncloud)
2. The vmail user
I want to use dovecot with virtual users. Now the question is: shouldn't
the directory be accessed only by the vmail user? and not by the owncloud
user?
Second Quest:
If it's correct that the access is made by 2 users: what rights do they
need?
I Cant add all users to the group vmail and set g+rwx Permissions (every
user could read mails from other users, and even edit them!)
Thanks a Lot.
Kind Regards,
Marcel
--
dovecot --version
2.2.13
dovecot -n
# 2.2.13: /etc/dovecot/dovecot.conf
# OS: Linux 3.13.0-37-generic x86_64 Ubuntu 14.04.1 LTS ext4
auth_mechanisms = plain login
auth_verbose = yes
info_log_path = /var/log/dovecot-info.log
log_path = /var/log/dovecot.log
mail_home = /home/vmail/example.com/%n
mail_location = maildir:/home/vmail/example.com/%n/mail:LAYOUT=fs
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags
copy include variables body enotify environment mailbox date ihave duplicate
namespace inbox {
inbox = yes
location =
mailbox Drafts {
auto = subscribe
special_use = \Drafts
}
mailbox Junk {
auto = subscribe
special_use = \Junk
}
mailbox Sent {
auto = subscribe
special_use = \Sent
}
mailbox Sent Messages {
auto = subscribe
special_use = \Sent
}
mailbox Trash {
auto = subscribe
special_use = \Trash
}
prefix =
}
passdb {
args = username_format=%u scheme=ssha512 /etc/dovecot/passwd.db
driver = passwd-file
}
plugin {
sieve = ~/.dovecot.sieve
sieve_after = /home/vmail/sieve-after
sieve_before = /home/vmail/sieve-before
sieve_dir = ~/sieve
}
protocols = imap sieve
service auth {
unix_listener /var/spool/postfix/private/dovecot-auth {
group = postfix
mode = 0660
user = postfix
}
}
ssl_cert = /etc/ssl/certs/ssl_main.crt
ssl_cipher_list =
ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AES:RSA+3DES:!ADH:!AECDH:!MD5:!DSS
ssl_client_ca_dir = /etc/ssl/certs
ssl_key = /etc/ssl/private/ssh_main_insecure
userdb {
args = uid=5000 gid=5000 home=/home/vmail/example.com/%n
driver = static
}
protocol imap {
imap_client_workarounds = delay-newmail tb-extra-mailbox-sep
mail_max_userip_connections = 10
}
protocol lda {
deliver_log_format = msgid=%m: %$
mail_plugins = sieve
postmaster_address = postmas...@example.com
quota_full_tempfail = yes
rejection_reason = Your message to %t was automatically rejected:%n%r
}
Migrating to Dovecot EE
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hello everyone! I am currently running dovecot 2.2.9 on my Debian Wheezy system. As this version is a bit outdated and I'm not too keen on compiling Dovecot myself I was thinking about moving to the Dovecot EE repository. I already have access to the repository, and now before I starting apt-getting all kinds of packages: What would be the best way on moving to the Dovecot EE repository? Just doing an apt-get install dovecot-ee? Will it replace the current version? Or should the old version be removed before installing the new one? Has anyone ever done this and has some advice for me? Thanks in advance, Philipp -BEGIN PGP SIGNATURE- iQEcBAEBAgAGBQJUN50yAAoJEC6sCbYfpzg9ilcH/iREEyhyygKEVrCyLRjutcLm iLa63KHCFGFjU3b5igZFD+Jv1uZmHfyLh+ZFz0ditHmS1jbNE49WKGawAuoBb3lv VqMCex3qsOA9OVdAbPYUCecs7K18Oo8DjwFpI4Sn70flsaFpo3JS32I/dZ9OMF7d p9O/G1PfJ4mw4zWC0HGgE7kluYytPBJTw2rqqAyjHXypQhcpiF+HHtFm/pX9SF4U b4K5YqfFJE/Bk7YYEdnQFyribcDxkllIzeAt09EhpEwZHK8MTWi92PwnHDcTX3M4 P7JnmSO3YtfSphbqDOFZgAYz2otEAPgysXu3v4asLkcoiPQb/H7hUlcNZyzLeIo= =fu6j -END PGP SIGNATURE-
Re: Migrating to Dovecot EE
Philipp Resch phil...@devh.de wrote: I am currently running dovecot 2.2.9 on my Debian Wheezy system. As this version is a bit outdated and I'm not too keen on compiling Dovecot myself I was thinking about moving to the Dovecot EE repository. I already have access to the repository, and now before I starting apt-getting all kinds of packages: What would be the best way on moving to the Dovecot EE repository? Just doing an apt-get install dovecot-ee? Will it replace the current version? Or should the old version be removed before installing the new one? Has anyone ever done this and has some advice for me? I've done it like this: 1) backup /etc/dovecot 2) purge old dovecot packages This removes all dovecot internal users, but do not worry, they will be recreated by the dovecot-ee packages. This also deletes all config files, but you have a backup from step 1. 3) install dovecot-ee packages 4) put config files from backup back to /etc/dovecot Beware: the pacakge system allows you to replace the old dovecot packages with the dovecot-ee packages, but due to the way the Debian package system works, this will wreck your dovecot-configuration once you finally purge the old packages, because then this also purges the old config files and deletes the dovecot-internal users (like dovenull). This is why it is very important to _first_ delete _and_ purge the old packages before installation of the dovecot-ee packages. Grüße, Sven. -- Sigmentation fault. Core dumped.
Re: question: dovecot lda running as special user (vmail) or normal user (marcel)?
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On Fri, 10 Oct 2014, marcel.cochem wrote:
I'm new to dovecot and wanted to build my own mailserver using
dovecot+postfix.
I currently have a Problem with Permissions with my mailfolder.
I could solve it easily if i set it with chmod to 777. but thats no real
solution.
While reading my log files i see a lot messages like:
Oct 10 05:19:52 lda(owncloud): Error: user owncloud: Initialization failed:
Initializing mail storage from mail_location setting failed:
stat(/home/vmail/example.com/owncloud/mail) failed: Permission denied
(euid=100(owncloud) egid=1004(owncloud) missing +x perm: /home/vmail, dir
owned by 5000:5000 mode=0700)
Oct 10 05:19:52 lda(owncloud): Fatal: Invalid user settings. Refer to
server log for more information.
So Currently two Users need to acces the mail-folder:
1. The user itself (here: owncloud)
2. The vmail user
I want to use dovecot with virtual users. Now the question is: shouldn't
the directory be accessed only by the vmail user? and not by the owncloud
user?
postfix starts the LDA as owncloud user, looks like you set postfix up to
use system users. Either make it use the vmail user or use LMTP.
userdb {
args = uid=5000 gid=5000 home=/home/vmail/example.com/%n
driver = static
}
Via IMAP/POP3 and LMTP all users will use these settings.
- --
Steffen Kaiser
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (GNU/Linux)
iQEVAwUBVDemh3z1H7kL/d9rAQIMxwf/cLh7M/VVUzweg5Vg1hQZx5rneCh/Lj6l
7NK1HSwEaXz/4u6kVQOpqXXRUhjHz9DXgAh6blQDifXOVHY3V1MpDleg8DKzHeah
wmhnfw3jWNuAGWd7z96Iys8mjuopPz35hy6nhVVwQtmv3wbFdqqch4PrCa2pOnvc
gnWUaht2wSdHhAP4ZCyNY5zWPPEQtIigpXqcYpfIRXVg/wO9TiYH0Uww6BBZvNUl
W/LRlS2E0jlQeacAueyRcdjoMuKC0ki/2ao12GWBffGW+2EJTD8U6dSTu4Ogviyr
g8SsPqUIG4NpQAvB2T5XwjECmsiR2gkxYaEJ8tLki0vUj4iQDGFgOA==
=aeeT
-END PGP SIGNATURE-
Re: dovecot replication (active-active) - server specs
Hello Am 09.10.2014 um 20:41 schrieb Urban Loesch: Hi, Am 09.10.2014 12:35, schrieb Martin Schmidt: Our MX server is delivering ca. 30 GB new mails per day. Two IMAP proxy server get the connections from the users. Atm. without dovecot director. We've got around 700k connections per day (imap 200k / pop3 500k) Are this the hole connections per day? How many concurrend connections do you have at the same time on each server? we've got 3 Fileserver with ca. 1200 concurrend IMAP connections and ca. 50 concurrend POP3 connections on each server. So we want to make a new system. We desire the new system to use mdbox format ( bigger files, less I/O) and replication through dovecot replication (active/active) instead of drbd. I have no experience with dovecot replication (Still on our roadmap). We are currently using drbd on a 10Gbit dedicated link. Works very well for us. Each fileserver should know every mailbox/user and for the time being 2 dovecot proxies for the user connections (IMAP/POP). (later after the migration from the old system to the new, dovecot director instead of proxies, for caching reasons). As Florian said, enable zlib. This also decreases I/O, but needs a bit more of CPU. But not that much. Yes we have enabled it, estimated space saving is up to 40% we've got 2 new fileservers, they have each SSD HDDs for new-storage and 7200rpm SATA HDDs on RAID 5 with 10 TB for alt-storage 32 GB RAM per Server You also could move the INDEX files from mdbox to different SSDs. We are doing so with 40k accounts and 2TB user data. Index partition has only 22GB used and is increasing not very fast. On our testsystem we've got it also on a raid 1 SSD, only alt-storage is on raid 5. Looks good. Do you have some tips for the system? Do you believe 32 GB RAM are enough for one fileserver each and have you experience with the I/O Waiting problem with huge amounts of Data on the alt-storage? Could there be issues with the RAM, if one fileserver has a downtime, so the second one has to take over all mailboxes for a short amount of time? I think memory is not the problem. On IMAP/POP3 servers the main problem is I/O. But with dovecot mdbox and index files on SSD's we have no problem at the moment. On each of our 3 Fileserver we've got 16 GB RAM, 5-7 GB is used and rest is cached. You might be right, the i/o is always the bottleneck. In general are only 2 new fileserver enough or should we think in bigger dimensions, like 4 fileserver Storage expansion in the new servers should not be a problem (bigger HDDs and a few slots free, so we can expand the raid 5). We are using raid 10 hardware raid controller with cache and sata 7200rpm disks. OK, raid 10 needs more disks, but is much faster than raid 5. Raid 5 is not very fast in my eyes. We've made tests with raid 10 and raid 5, on 4 sata 7200rpm disks, of course raid 10 was faster, but overall not very much. And you can expand raid 5 easier. Can you tell me, if you have a high Waiting on your alt-storage? thank you kind regards Martin Schmidt Regards Urban Thank you for your impressions. kr Martin Schmidt
Re: index problem with only 1 folder of 1 box
Hello, I retry. Nobody does have an idea about my index problem? The biggest trouble for me is : Is it a solr problem or a dovecot problem? In my opinion, it's more a dovecot problem because the first research after a solr reindex give the good answer. Can someone explains to me how dovecot creates and uses cache please? Once again, sorry for my poor english level. Thank you in advance Guillaume Le 07/10/2014 12:23, Guillaume a écrit : Hello, First of all, sorry for my poor english level. I pass to solr indexing. All is OK, except for the inbox of one of the mailboxes. For this inbox (other folders of the same mailbox have no problems), when i do text search, there is always no response. Others mailboxes have no problems. If i do a : doveadm -Dv fts rescan -u my...@domain.tld, i can do one and only one search with the correct response. It index the folder, take a few seconds and give me the good answer. If i redo the same search, it's extremly quick with no answers. It's like a cache problem but i don't know where and why... Logs are not very verbose... Here is an example of command line search : $ nc 127.0.0.1 143 * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE STARTTLS AUTH=PLAIN AUTH=LOGIN] Bienvenue sur le serveur IMAP de Domain.tld 10 LOGIN my...@domain.tld MyBigSecret 10 OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE BINARY MOVE SEARCH=FUZZY QUOTA] Logged in 20 SELECT INBOX * FLAGS (\Answered \Flagged \Deleted \Seen \Draft unknown-2 unknown-9 unknown-0 unknown-10 unknown-1 unknown-4 $MDNSent unknown-3 unknown-8 $Forwarded NonJunk) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft unknown-2 unknown-9 unknown-0 unknown-10 unknown-1 unknown-4 $MDNSent unknown-3 unknown-8 $Forwarded NonJunk \*)] Flags permitted. * 8437 EXISTS * 0 RECENT * OK [UIDVALIDITY 1412669180] UIDs valid * OK [UIDNEXT 8439] Predicted next UID * OK [HIGHESTMODSEQ 20] Highest 20 OK [READ-WRITE] Select completed (0.000 secs). 30 SEARCH text coucou * OK Indexed 14% of the mailbox, ETA 0:57 * OK Indexed 29% of the mailbox, ETA 0:47 * OK Indexed 72% of the mailbox, ETA 0:11 * OK Mailbox indexing finished * SEARCH 35 108 153 175 176 182 348 357 360 431 472 479 520 574 575 576 577 578 586 949 984 1025 1032 1033 1034 1087 1099 1129 1284 1313 1314 1315 1366 1372 1465 1538 1711 1726 1758 1759 1969 1981 1983 1984 2036 2077 2104 2116 2162 2181 2193 2437 2504 2587 2663 2808 2809 2854 2861 2862 3076 3087 3088 3137 3188 3189 3190 3191 3192 3193 3196 3197 3250 3251 3252 3327 3328 3329 3330 3331 3332 3379 3464 3481 3494 3521 3528 3582 3589 3593 3596 3651 3667 3672 3695 3787 3789 3939 3940 3941 3942 3943 3944 3945 3946 4019 4020 4034 4043 4134 4146 4175 4241 4346 4379 4422 4490 4491 4537 4540 4542 4543 4544 4550 4627 4727 4742 4767 4790 4817 4828 4831 4839 4862 4888 4899 4901 4906 4911 4916 4957 4999 5003 5004 5008 5009 5010 5011 5012 5013 5039 5062 5081 5092 5119 5138 5145 5150 5152 5185 5186 5187 5216 5246 5247 5282 5283 5314 5316 5320 5322 5325 5337 5353 5360 5363 5385 5432 5447 5472 5485 5496 5498 5500 5528 5551 5564 5567 5595 5598 5642 5794 5841 5896 5933 5960 5961 5994 6001 6005 6016 6017 6051 6075 6125 6152 6163 6215 6238 6239 6240 6241 6287 6311 6342 6350 6353 6357 6369 6431 6472 6689 6928 6945 6951 7026 7030 7045 7046 7115 7210 7212 7214 7230 7256 7295 7298 7340 7489 7490 7492 7654 7655 7658 7660 7676 7767 7768 7800 7801 7913 7937 7956 8075 8085 8102 8111 8144 8266 8329 8333 8394 8395 8431 30 OK Search completed (36.029 secs). 40 SEARCH text coucou * SEARCH 40 OK Search completed (0.002 secs). 50 SEARCH text blabla * SEARCH 50 OK Search completed (0.002 secs). 60 LOGOUT * BYE Logging out 60 OK Logout completed. To close this question, here is dovecot -n: # 2.2.9: /etc/dovecot/dovecot.conf # OS: Linux 3.13.0-36-generic x86_64 Ubuntu 14.04.1 LTS ext4 auth_mechanisms = plain login debug_log_path = /var/log/dovecot-debug.log disable_plaintext_auth = no first_valid_uid = 5000 hostname = mailserver.domain.tld info_log_path = /var/log/dovecot-info.log last_valid_uid = 5000 lda_mailbox_autocreate = yes lda_original_recipient_header = X-Original-To listen = * log_path = /var/log/dovecot-err.log login_greeting = Bienvenue sur le serveur IMAP de Domain.tld mail_debug = yes mail_gid = 5000 mail_location = maildir:/data/mails/%d/%n mail_plugins = quota fts fts_solr mail_uid = 5000 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body
Re: index problem with only 1 folder of 1 box
Hello On 10/10/2014 02:50 PM, Guillaume wrote: The biggest trouble for me is : Is it a solr problem or a dovecot problem? In my opinion, it's more a dovecot problem because the first research after a solr reindex give the good answer. If at all possible, you should try a newer version of Dovecot. There have been quite a few changes to FTS and the SOLR backend since version 2.2.9. See attached log for HG log of changes. Hope this helps. br, Teemu Huovila 2014-09-16 15:23 +0300 Timo Sirainen t...@iki.fi (8c2cb7d01a78) * src/plugins/fts-solr/fts-backend-solr.c, src/plugins/fts/fts-api- private.h, src/plugins/fts/fts-api.c: doveadm fts rescan: For virtual namespaces just mark the last indexed UID to 0. 2014-09-16 14:32 +0300 Timo Sirainen t...@iki.fi (e82ad7f1c58f) * src/plugins/fts/fts-expunge-log.c: fts: dovecot-expunges.log wasn't closed at deinit 2014-08-08 16:27 +0300 Timo Sirainen t...@iki.fi (1ea3da40ea8f) * src/plugins/fts/fts-storage.c: fts: fts_no_autofuzzy shouldn't disable fuzzying when FUZZY search parameter is set. 2014-08-08 16:20 +0300 Timo Sirainen t...@iki.fi (cdf4edcc6256) * src/plugins/fts-lucene/fts-backend-lucene.c, src/plugins/fts-lucene /lucene-wrapper.cc, src/plugins/fts-lucene/lucene-wrapper.h, src/plugins/fts-solr/fts-backend-solr-old.c, src/plugins/fts-solr /fts-backend-solr.c, src/plugins/fts-squat/fts-backend-squat.c, src/plugins/fts/fts-api-private.h, src/plugins/fts/fts-api.c, src/plugins/fts/fts-api.h, src/plugins/fts/fts-search.c, src/plugins/fts/fts-storage.c, src/plugins/fts/fts-storage.h: fts: Added fts_no_autofuzzy setting to require exact matches for found results. This is done by using the FTS search results as only filters on which the regular non-FTS search is done. 2014-07-03 14:37 +0300 Timo Sirainen t...@iki.fi (9c6643daae98) * src/plugins/fts/fts-expunge-log.c: fts: If we detect corrupted fts expunge log, unlink it. This avoids the same error repeating forever. 2014-06-30 17:25 +0300 Timo Sirainen t...@iki.fi (2c2b94840ff3) * src/plugins/fts/fts-parser-tika.c: fts-tika: Hiden Unsupported Media Type errors. Log HTTP status code on errors. 2014-06-30 16:41 +0300 Timo Sirainen t...@iki.fi (49dfc6da1786) * src/plugins/fts/fts-parser-tika.c: fts-tika: Fixed crash if Tika returned 200 reply without payload. 2014-06-16 15:35 +0300 Timo Sirainen t...@iki.fi (fc40b1a6e962) * src/plugins/fts/xml2text.c: xml2text: Check for read()/write() failures and exit if they fail. 2014-06-13 02:19 +0300 Timo Sirainen t...@iki.fi (b67c1c9bf1a5) * src/auth/mech-winbind.c, src/auth/userdb-passwd-file.c, src/config /config-parser.c, src/doveadm/doveadm-director.c, src/doveadm /doveadm-dump-dbox.c, src/doveadm/doveadm-log.c, src/doveadm /doveadm-penalty.c, src/doveadm/doveadm-replicator.c, src/doveadm /doveadm-stats.c, src/doveadm/doveadm-who.c, src/doveadm/doveadm- zlib.c, src/lib-compression/test-compression.c, src/lib-imap-urlauth /imap-urlauth-connection.c, src/lib-lda/smtp-client.c, src/lib- master/master-instance.c, src/lib-master/mountpoint-list.c, src/lib- settings/settings-parser.c, src/lib-settings/settings.c, src/lib- storage/index/cydir/cydir-mail.c, src/lib-storage/index/dbox-common /dbox-file.c, src/lib-storage/index/imapc/imapc-save.c, src/lib- storage/index/maildir/maildir-mail.c, src/lib-storage/index/raw/raw- storage.c, src/lib-storage/list/subscription-file.c, src/lib /iostream-temp.c, src/lib/istream-seekable.c, src/plugins/fts/fts- parser-script.c, src/plugins/zlib/zlib-plugin.c, src/replication/replicator/replicator-queue.c, src/ssl- params/main.c, src/util/rawlog.c: Use the new [io]_stream_create_fd_*autoclose() functions wherever possible. 2014-06-13 01:11 +0300 Timo Sirainen t...@iki.fi (54f1beb8d071) * src/plugins/fts/doveadm-dump-fts-expunge-log.c: fts: Improved doveadm fts dump for corrupted expunge log Although we may still be trying to allocate up to 2 GB of memory, but at least no more than that now. Found by Coverity 2014-06-13 00:46 +0300 Timo Sirainen t...@iki.fi (0fc86de05ccf) * src/plugins/fts/doveadm-dump-fts-expunge-log.c: fts: Minor code cleanup: Don't increment NULL pointer. 2014-05-27 21:17 +0300 Phil Carmody p...@dovecot.fi (ad028a950248) * src/plugins/fts/fts-parser-html.c: fts: parser-html - parser can fail on attributes='with values in single quotes' If that value were to contain an odd number of double quotes, then the HTML_STATE_TAG_(D)QUOTED state would be entered and not exited. The two quoting types behave basically the same, so just add two new cases and duplicate the state transition code. 2014-05-27 21:17 +0300 Phil Carmody p...@dovecot.fi (54e508b71dcd) * src/plugins/fts/fts-parser-html.c: fts: parser-html - parse_tag_name returns wrong value for comments This function returns 1 more than the number of additional characters to be swallowed up by the state transition. 2014-05-27 21:17 +0300 Phil
Re: Question wrt. dovecot replicator
On Fri, 2014-10-03 at 23:37 +0200, Remko Lodder wrote: How can I determine why there are duplicated emails? Same problem here! What kind of messages should I specifically look for? Look for any errors and warnings in the Dovecot log. You could also enable mail_debug (ref. Can I set this up for a few selected accounts instead of all accounts like it was currently? To make sure I do not make things worse for others then needs to be :-) The service had been disabled for the time being to prevent the other users from getting duplicated emails. I do not know what kind of userdb you are running, but there is a newish patch that enables per user replication via the mail_replica setting. It is not yet included in the newest (2.2.13) release of Dovecot, but is available via the enterprise version. There are no FreeBSD builds for that, though. ref: http://hg.dovecot.org/dovecot-2.2/rev/c1c67bdc8752 my userdb consists of local users (Which are fed through LDAP at the backend). perhaps I can setup a mailAttributes setting or something so that the replica can be set, although I prefer that I have control over that in the config itself :-) With the latest Debian jessie version 1:2.2.13-5 you can actually have a per user mail_replica setting taken from a (LDAP) directory. This keeps the duplicate mail issue away from other users but a few willing to test ... For what it’s worth: replication_dsync_parameters = -f -d -N -l 30 -U I read in Peer Heinlein's Dovecot book http://www.opensourcepress.de/de/produkte/Dovecot/13560/978-3-95539-074-7 that replicating a public namespace gives you trouble :-(. So keep the -N option away for now. However I would like examples for the -n and -x options, which are neither given in the wiki nor in the nonexistant man page. I think the replication feature is very, very cool, but right now it gives me a hard time to implement flawlessly ;-) Thanks for any help or light shed on this issue ... Cheers Jan -- MAX-PLANCK-INSTITUT fuer Radioastronomie Jan Behrend - Rechenzentrum Auf dem Huegel 69, D-53121 Bonn Tel: +49 (228) 525 359, Fax: +49 (228) 525 229 jbehr...@mpifr-bonn.mpg.de http://www.mpifr-bonn.mpg.de signature.asc Description: This is a digitally signed message part smime.p7s Description: S/MIME cryptographic signature
Re: index problem with only 1 folder of 1 box
Thank you for your answer. I will try to upgrade dovecot. By the way, I erase cache of solr ( deletequery*:*/query/delete ) and the problem is always here... When I will can upgrade dovecot, i will write back to say if its good or not. Sincerely yours, Guillaume Le 10/10/2014 14:15, Teemu Huovila a écrit : Hello On 10/10/2014 02:50 PM, Guillaume wrote: The biggest trouble for me is : Is it a solr problem or a dovecot problem? In my opinion, it's more a dovecot problem because the first research after a solr reindex give the good answer. If at all possible, you should try a newer version of Dovecot. There have been quite a few changes to FTS and the SOLR backend since version 2.2.9. See attached log for HG log of changes. Hope this helps. br, Teemu Huovila
Re: Dbox and Exim
On October 10, 2014 10:10:55 AM Robert Schetterer r...@sys4.de wrote: sorry german Why sorry ?, its more sad to see it repeated Men måske skulle jeg bare skrive blåbærgrød ? :)
fixes for quota support on NetBSD
Hi!
dovecot-2.2.13 already has quota support for NetBSD, but it's buggy.
The attached patches by Manuel Bouyer bou...@netbsd.org fix the
issues.
There is one thing that's not nice in them: one include is now for
/usr/include/quota.h since dovecot comes with its own file quota.h
which is earlier in the search path. Perhaps dovecot's copy can be
renamed to dovecot-quota.h or to some other non-conflicting name?
Cheers,
Thomas
$NetBSD: patch-src_plugins_quota_quota-fs.c,v 1.4 2013/08/05 23:12:42 bouyer
Exp $
fix support for NetBSD's libquota
--- src/plugins/quota/quota-fs.c.orig 2013-02-26 09:42:04.0 +0100
+++ src/plugins/quota/quota-fs.c2013-08-06 01:00:32.0 +0200
@@ -672,34 +672,38 @@
{
struct quotakey qk;
struct quotaval qv;
+ struct quotahandle *qh;
+ int ret;
- if (root-qh == NULL) {
- if ((root-qh = quota_open(root-mount-mount_path)) == NULL) {
- i_error(cannot open quota for %s: %m,
- root-mount-mount_path);
- fs_quota_root_disable(root, group);
- return 0;
- }
- }
+ if ((qh = quota_open(root-mount-mount_path)) == NULL) {
+ i_error(cannot open quota for %s: %m,
+ root-mount-mount_path);
+ fs_quota_root_disable(root, group);
+ return 0;
+ }
qk.qk_idtype = group ? QUOTA_IDTYPE_GROUP : QUOTA_IDTYPE_USER;
qk.qk_id = group ? root-gid : root-uid;
qk.qk_objtype = bytes ? QUOTA_OBJTYPE_BLOCKS : QUOTA_OBJTYPE_FILES;
- if (quota_get(root-qh, qk, qv) != 0) {
+ if (quota_get(qh, qk, qv) != 0) {
if (errno == ESRCH) {
fs_quota_root_disable(root, group);
return 0;
}
i_error(quotactl(Q_GETQUOTA, %s) failed: %m,
root-mount-mount_path);
- return -1;
+ ret = -1;
+ goto end;
}
*value_r = qv.qv_usage * DEV_BSIZE;
*limit_r = qv.qv_softlimit * DEV_BSIZE;
- return 1;
+ ret = 1;
+end:
+ quota_close(qh);
+ return ret;
}
#endif
$NetBSD: patch-src_plugins_quota_quota-fs.h,v 1.4 2013/08/05 23:12:42 bouyer
Exp $
fix support for NetBSD's libquota
--- src/plugins/quota/quota-fs.h.orig 2013-08-06 00:53:34.0 +0200
+++ src/plugins/quota/quota-fs.h2013-08-06 00:54:29.0 +0200
@@ -7,7 +7,8 @@
#endif
#ifdef HAVE_QUOTA_OPEN
-# include quota.h /* NetBSD with libquota */
+/* absolute path to avoid confusion with ./quota.h */
+# include /usr/include/quota.h /* NetBSD with libquota */
#endif
#ifdef HAVE_SYS_QUOTA_H
Re: Dbox and Exim
Am 10.10.2014 um 15:38 schrieb Benny Pedersen: On October 10, 2014 10:10:55 AM Robert Schetterer r...@sys4.de wrote: sorry german Why sorry ?, its more sad to see it repeated Men måske skulle jeg bare skrive blåbærgrød ? :) jIyajbe' Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
Error with dovecot 2.2.14rc1
I don't know if this is significant or not, but it is at least unusual here. First we see this: Oct 9 10:10:03 sbh16 dovecot: imap(mark): Error: Sync failed for mbox file /var/spool/mail/mark: seq=393 uid=201517 uid_broken=0 originally needed 0 bytes, now needs 66 bytes which has apparently added 66 bytes to UID 201517 Then, later we see: Oct 9 11:26:19 sbh16 dovecot: imap(mark): Error: Cached message size larger than expected (32353 32287) Oct 9 11:26:19 sbh16 dovecot: imap(mark): Error: Corrupted index cache file /home/mark/Mail/.imap/INBOX/dovecot.index.cache: Broken physical size for mail UID 201517 Oct 9 11:26:19 sbh16 dovecot: imap(mark): Error: read(/var/spool/mail/mark) failed: Cached message size larger than expected (32353 32287) (FETCH BODY[] for mailbox INBOX UID 201517) Oct 9 11:26:19 sbh16 dovecot: imap(mark): Disconnected: FETCH read() failed in=10948 out=1206206 As of now (almost 22 hours later) these are the only dovecot Error messages in the log. Going back a full month, we see this under dovecot 2.2.13 Sep 9 10:18:01 sbh16 dovecot: imap(mark): Error: Sync failed for mbox file /var/spool/mail/mark: seq=198 uid=198968 uid_broken=0 originally needed 0 bytes, now needs 66 bytes but no subsequent errors like yesterday. -- Mark Sapiro m...@msapiro.netThe highway is for gamblers, San Francisco Bay Area, Californiabetter use your sense - B. Dylan
Re: 2.2.14rc1 - dsync in backup mode still changes source permissions
On 10 Oct 2014, at 11:05, Peter Mogensen a...@one.com wrote: It seems we are still able to reproduce this: http://www.dovecot.org/list/dovecot/2014-May/096367.html However... there's no longer any error-messages. It just silently changes permissions on some dovecot files in the source maildir. (most often dovecot-uidlist) We're running dsync as root, with hardwired userdb values for other reasons. So it has the OS permissions to change source. But still, running in backup shouldn't change source ever, should it? It's not doing any changes to mailbox contents, but it's still updating the index/uidlist files as part of its normal operation. The command line is of this format - running on destination-host: # dsync -R -o mail_home=/users/user/maildir backup ssh -c arcfour -o StrictHostKeyChecking=no -i /root/.ssh/id-rsa-dsync source-host dsync -o mail_home=/users/user/maildir You should use -u user@domain parameter in both sides so it drops root privileges.
Re: 2.2.14rc1 - dsync in backup mode still changes source permissions
On 11 Oct 2014, at 00:52, Timo Sirainen t...@iki.fi wrote: On 10 Oct 2014, at 11:05, Peter Mogensen a...@one.com wrote: It seems we are still able to reproduce this: http://www.dovecot.org/list/dovecot/2014-May/096367.html However... there's no longer any error-messages. It just silently changes permissions on some dovecot files in the source maildir. (most often dovecot-uidlist) We're running dsync as root, with hardwired userdb values for other reasons. So it has the OS permissions to change source. But still, running in backup shouldn't change source ever, should it? It's not doing any changes to mailbox contents, but it's still updating the index/uidlist files as part of its normal operation. The command line is of this format - running on destination-host: # dsync -R -o mail_home=/users/user/maildir backup ssh -c arcfour -o StrictHostKeyChecking=no -i /root/.ssh/id-rsa-dsync source-host dsync -o mail_home=/users/user/maildir You should use -u user@domain parameter in both sides so it drops root privileges. Oh, and reading the linked mail more closely, if the maildir S=sizes have problems then Dovecot attempts to fix them. It's the same as if you attempted to read the mails via any method. doveadm backup doesn't attempt to read the whole source maildir without any modifications, although it could, but that could cause performance problems. Anyway, if you have broken S=sizes, you could try setting maildir_broken_filename_sizes=yes.
