Re: [Dovecot] Recalculate quota when quota=dict ?

2012-02-21 Thread Timo Sirainen
Maybe http://hg.dovecot.org/dovecot-2.1/rev/2ea29ab6f10f fixes it?

On Tue, 2012-02-21 at 08:01 +0100, Javier Miguel Rodríguez wrote:
 I have seen this behaviour with a local ext4 iSCSI filesystem. When the 
 system is hammered by I/O (example, perfoming a full backup), I also see 
 those messages in the log.
 
 Regards
 
 Javier
 
 
  On 17.2.2012, at 11.51, jos...@hybrid.pl wrote:
 
  By the way: what might have caused such a warning?
 
  r...@mail2.hybrid.pl /tmp/transfer  doveadm quota recalc -u 
  jos...@hybrid.pl
  doveadm(jos...@hybrid.pl): Warning: Created dotlock file's timestamp is 
  different than current time (1329464622 vs 1329464672): 
  /var/mail/mail/hybrid.pl/joshua/.mailing.ekg/dovecot-uidlist
  Does it keep happening? Is this a local filesystem or NFS? Shouldn't happen 
  unless remote storage server's clock and local server's clock aren't synced.
 
 




Re: [Dovecot] 2.1 upgrade docs?

2012-02-21 Thread Juan C. Blanco



On 21/02/2012 1:17, Timo Sirainen wrote:

On 20.2.2012, at 18.43, Juan C. Blanco wrote:


I'm just migrating one of our servers to 2.1 and noticed a possible 
configuration problem that I've missed to detect with the RCs.

You ca have a quota problema if you have a per-user quota based on an ldap 
attribute and if your configuration file dovecot-ldap.conf.ext uses an old 
format; if the user attributes line looks like

user_attrs=,ldap-attribute=quota_rule=*:storage


This was unintentional. Fixed: 
http://hg.dovecot.org/dovecot-2.1/rev/e8da2675da41


OK, thanks Timo even if the workaround was not difficult at all





--
++
| Juan C. Blanco |
||
|  Centro de Calculo  |  |
|  Facultad de Informatica U.P.M. |  E-mail: jcbla...@fi.upm.es  |
|  Campus de Montegancedo |  |
|  Boadilla del Monte |  Tel.:(+34) 91 336 7466  |
|  28660 MADRID (Spain)   |  Fax :(+34) 91 336 6913  |
++


Re: [Dovecot] Some trouble with dovecot 2.1 and SASL

2012-02-21 Thread Timo Sirainen
On Sun, 2012-02-19 at 11:39 +0100, Alessio Cecchi wrote:

 I'm testing dovecot 2.1.0, on Debian 6.0, with passdb driver = imap 
 and SASL for postfix's SMTP-Auth, like explained ih this howto: 
 http://wiki2.dovecot.org/HowTo/PostfixAndDovecotSASL
 
 Authentications over IMAP (local dovecot - remote dovecot) works fine, 
 and it's great. Also smtp-auth on postfix (over SASL) works fine. But in 
 the dovecot log I found this error:
 
 
 Feb 19 11:11:42 demo-vpop dovecot: auth: Fatal: master: service(auth): 
 child 30506 killed with signal 11 (core dumps disabled)

http://hg.dovecot.org/dovecot-2.1/rev/a6a58512f0df should fix this.




[Dovecot] passwd-file - lower limit for username?

2012-02-21 Thread Oon-Ee Ng
Hi, previously I could use the following in my passwd-file:-

AB:{plain}password

I'm not sure when it stopped working, its not an account I access with
any regularity, but today it would not work (2.1.0, 64-bit Arch
Linux), with an 'unknown user' error in the logs. Is there some new
lower limit for username length?


Re: [Dovecot] Folder name migration when moving from 2.0.x to 2.1.x?

2012-02-21 Thread Ralf Hildebrandt
* Timo Sirainen t...@iki.fi:
 On 18.2.2012, at 19.18, Ralf Hildebrandt wrote:
 
  Feb 18 18:15:40 postamt dovecot: imap(lehnertr): Panic: LIST: Mailbox name 
  not UTF-8: INBOX.Entw�rfe
 
 http://hg.dovecot.org/dovecot-2.1/rev/7d0d76df719f should fix this by 
 removing the invalid entry from subscriptions file (probably broken client 
 added it in some earlier Dovecot version that didn't disallow it).

I think that's very likely. I'll update later on. Thanks again.

-- 
Ralf Hildebrandt
  Geschäftsbereich IT | Abteilung Netzwerk
  Charité - Universitätsmedizin Berlin
  Campus Benjamin Franklin
  Hindenburgdamm 30 | D-12203 Berlin
  Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962
  ralf.hildebra...@charite.de | http://www.charite.de



Re: [Dovecot] passwd-file - lower limit for username?

2012-02-21 Thread Timo Sirainen
On Tue, 2012-02-21 at 17:33 +0800, Oon-Ee Ng wrote:
 Hi, previously I could use the following in my passwd-file:-
 
 AB:{plain}password
 
 I'm not sure when it stopped working, its not an account I access with
 any regularity, but today it would not work (2.1.0, 64-bit Arch
 Linux), with an 'unknown user' error in the logs. 

Dovecot doesn't think the user exists. auth_debug=yes may be helpful,
and doveconf -n output.

 Is there some new lower limit for username length?

No.




Re: [Dovecot] [dovecot] dovecot freezes when trying to get mail from maildir with mail

2012-02-21 Thread postmaster
[ I'm sorry I can't reply in thread, but at present I'm reading email as 
plain text files directly off the file system - which is a PITA ;-) ]


Bjarne D Mathiesen
København N ; Danmark ; Europa
++
denne besked er skrevet i et totalt M$-frit miljø
MacOS X 10.7.3 Lion ; 2.8GHz Intel Core i7 ; 16GB 1067MHz DDR3

On a note regarding my hardware :
the computer has three ethernet-interfaces:
but that doesn't really seem to be an issue as telnet from localhost 
also hangs indefinetely
I'm seeing the issue no matter which interface I connect to, whether the 
two external WAN or the internal LAN IP-number .


root# system_profiler SPEthernetDataType
2012-01-22 01:34:36.524 system_profiler[47377:903] Found bsdName
(0x2603) for AppleUSBEthernet
2012-01-22 01:34:36.626 system_profiler[47377:903] Found bsdName
(0x2c03) for AppleUSBEthernet
Ethernet Cards:

Apple USB Ethernet Adaptor:

  Type: USB
  BSD name: en4
  Kext name: AppleUSBEthernet.kext
  Location:
/System/Library/Extensions/IONetworkingFamily.kext/Contents/PlugIns/AppleUSBEthernet.kext
  Version: 2.1.39
  MAC Address: 00:1f:f3:8a:f3:5a
  Product ID: 5122
  Vendor ID: 1452

Apple USB Ethernet Adaptor:

  Type: USB
  BSD name: en3
  Kext name: AppleUSBEthernet.kext
  Location:
/System/Library/Extensions/IONetworkingFamily.kext/Contents/PlugIns/AppleUSBEthernet.kext
  Version: 2.1.39
  MAC Address: 00:1f:f3:8b:26:c1
  Product ID: 5122
  Vendor ID: 1452

Marvell Yukon Gigabit Adapter 88E8053 Singleport Copper SA:

  Name: ethernet
  Type: Ethernet Controller
  Bus: PCI
  Vendor ID: 0x11ab
  Device ID: 0x4362
  Subsystem Vendor ID: 0x11ab
  Subsystem ID: 0x5321
  Revision ID: 0x0022
  Link Width: x1
  BSD name: en0
  Kext name: AppleYukon2.kext
  Location:
/System/Library/Extensions/IONetworkingFamily.kext/Contents/PlugIns/AppleYukon2.kext
  Version: 3.2.1b1

ethernet:

  Type: Ethernet Controller
  Bus: PCI
  Slot: AirPort
  Vendor ID: 0x168c
  Device ID: 0x001c
  Subsystem Vendor ID: 0x106b
  Subsystem ID: 0x0086
  Revision ID: 0x0001
  Link Width: x1
  BSD name: en1
  Kext name: AirPortAtheros21.kext
  Location:
/System/Library/Extensions/IO80211Family.kext/Contents/PlugIns/AirPortAtheros21.kext
  Version: 425.14.7

root# ifconfig
lo0: flags=8049UP,LOOPBACK,RUNNING,MULTICAST mtu 16384
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
inet 127.0.0.1 netmask 0xff00
gif0: flags=8010POINTOPOINT,MULTICAST mtu 1280
stf0: flags=0 mtu 1280
en0: flags=8863UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST mtu 1500
ether 00:16:cb:a8:7d:01
inet6 fe80::216:cbff:fea8:7d01%en0 prefixlen 64 scopeid 0x4
inet 192.168.1.37 netmask 0xff00 broadcast 192.168.1.255
media: autoselect (1000baseT full-duplex,flow-control)
status: active
en4: flags=8863UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST mtu 1500
ether 00:1f:f3:8a:f3:5a
inet6 fe80::21f:f3ff:fe8a:f35a%en4 prefixlen 64 scopeid 0x5
inet 90.184.13.xxx netmask 0xfe00 broadcast 90.184.13.zzz
media: autoselect (100baseTX full-duplex,flow-control)
status: active
en3: flags=8863UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST mtu 1500
ether 00:1f:f3:8b:26:c1
inet6 fe80::21f:f3ff:fe8b:26c1%en3 prefixlen 64 scopeid 0x6
inet 90.184.12.yyy netmask 0xfe00 broadcast 90.184.13.zzz
media: autoselect (100baseTX full-duplex,flow-control)
status: active
fw0: flags=8863UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST mtu 2030
lladdr 00:17:f2:ff:fe:8e:e6:d2
media: autoselect full-duplex
status: inactive
en1: flags=8863UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST mtu 1500
ether 00:17:f2:52:62:9e
media: autoselect (unknown type)
status: inactive

root# system_profiler SPHardwareDataType
Hardware:

Hardware Overview:

  Model Name: Mac mini
  Model Identifier: Macmini1,1
  Processor Name: Intel Core Duo
  Processor Speed: 1,83 GHz
  Number Of Processors: 1
  Total Number Of Cores: 2
  L2 Cache: 2 MB
  Memory: 2 GB
  Bus Speed: 667 MHz
  Boot ROM Version: MM11.0055.B08
  SMC Version (system): 1.3f4

root# system_profiler SPSoftwareDataType
Software:

System Software Overview:

  System Version: Mac OS X 10.6.8 (10K549)
  Kernel Version: Darwin 10.8.0
  Boot Volume: X
  Boot Mode: Normal
  Computer Name: MiniWeb
  User Name: System Administrator (root)
  Secure Virtual Memory: Enabled
  64-bit Kernel and Extensions: No
  Time since boot: 5 days 1:31



On 19.2.2012, at 14.53, postmaster wrote:

 Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: auth-worker: Debug:
sql(ab...@mathiesen.info,127.0.0.1): SELECT  5000 AS uid , 

Re: [Dovecot] passwd-file - lower limit for username?

2012-02-21 Thread Timo Sirainen
On Tue, 2012-02-21 at 17:33 +0800, Oon-Ee Ng wrote:
 Hi, previously I could use the following in my passwd-file:-
 
 AB:{plain}password
 
 I'm not sure when it stopped working, its not an account I access with
 any regularity, but today it would not work (2.1.0, 64-bit Arch
 Linux), with an 'unknown user' error in the logs. Is there some new
 lower limit for username length?

Oh, I think it's because AB is uppercased. Either lowercase it ab or
set this to empty:

auth_username_format = 

I hadn't thought of this being a problem before.. Hm. Perhaps it's not a
common problem, so I'll just change the Upgrading text to mention this
possibility also..




Re: [Dovecot] [dovecot] dovecot freezes when trying to get mail from maildir with mail

2012-02-21 Thread Timo Sirainen
On Tue, 2012-02-21 at 10:58 +0100, postmaster wrote:
 
 #0  0x90003afa in mach_msg_trap ()
 No symbol table info available.
 #1  0x90004267 in mach_msg ()
 No symbol table info available.
 #2  0x9001b431 in vproc_mig_look_up2 ()
 No symbol table info available.
 #3  0x9001b26a in bootstrap_look_up2 ()
 No symbol table info available.
 #4  0x9001b0e4 in bootstrap_look_up ()
 No symbol table info available.
 #5  0x9001af30 in notify_register_check ()
 No symbol table info available.
 #6  0x9001ada7 in notify_register_tz ()
 No symbol table info available.
 #7  0x9001a621 in tzsetwall_basic ()
 No symbol table info available.
 #8  0x90041903 in localtime ()
 No symbol table info available.

All of the above functions are OSX's internal functions in libc.

 #9  0x0007ba2f in mail_index_update_day_headers ()

So Dovecot calls localtime() and the process hangs. Something's badly
broken in your system related to timezone handling, nothing Dovecot can
do about it..



Re: [Dovecot] [dovecot] dovecot freezes when trying to get mail from maildir with mail

2012-02-21 Thread postmaster
[ I'm sorry I can't reply in thread, but at present I'm reading email as 
plain text files directly off the file system - which is a PITA ;-) ]


Bjarne D Mathiesen
København N ; Danmark ; Europa
++
denne besked er skrevet i et totalt M$-frit miljø
MacOS X 10.7.3 Lion ; 2.8GHz Intel Core i7 ; 16GB 1067MHz DDR3

agh -- used pop3 instead of imap :-(

4604ds1-ynoe:~ root# gdb --args /macports/libexec/dovecot/imap -u 
postmas...@mathiesen.info
GNU gdb 6.3.50-20050815 (Apple version gdb-1518) (Sat Feb 12 02:49:13 
UTC 2011)

Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain 
conditions.

Type show copying to see the conditions.
There is absolutely no warranty for GDB.  Type show warranty for details.
This GDB was configured as i386-apple-darwin...Reading symbols for 
shared libraries . done


(gdb) run
Starting program: /macports/libexec/dovecot/imap -u 
postmas...@mathiesen.info

Reading symbols for shared libraries . done
Debug: auth input: postmas...@mathiesen.info uid=5000 gid=5000 
home=/Volumes/BrugerData/PostFix/mail/mathiesen.info/postmas...@mathiesen.info/
Debug: Effective uid=5000, gid=5000, 
home=/Volumes/BrugerData/PostFix/mail/mathiesen.info/postmas...@mathiesen.info/
Debug: maildir++: 
root=/Volumes/BrugerData/PostFix/mail/mathiesen.info/postmas...@mathiesen.info/, 
index=, control=, 
inbox=/Volumes/BrugerData/PostFix/mail/mathiesen.info/postmas...@mathiesen.info/, 
alt=
* PREAUTH [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID 
ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND 
UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE 
QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS] 
Logged in as postmas...@mathiesen.info

a select inbox
^C
Program received signal SIGINT, Interrupt.
0x90003afa in mach_msg_trap ()
(gdb) bt full
#0  0x90003afa in mach_msg_trap ()
No symbol table info available.
#1  0x90004267 in mach_msg ()
No symbol table info available.
#2  0x9001b431 in vproc_mig_look_up2 ()
No symbol table info available.
#3  0x9001b26a in bootstrap_look_up2 ()
No symbol table info available.
#4  0x9001b0e4 in bootstrap_look_up ()
No symbol table info available.
#5  0x9001af30 in notify_register_check ()
No symbol table info available.
#6  0x9001ada7 in notify_register_tz ()
No symbol table info available.
#7  0x9001a621 in tzsetwall_basic ()
No symbol table info available.
#8  0x90041903 in localtime ()
No symbol table info available.
#9  0x00092a2f in mail_index_update_day_headers ()
No symbol table info available.
#10 0x0008ec95 in mail_index_transaction_finish ()
No symbol table info available.
#11 0x0008d80a in mail_index_transaction_commit_v ()
No symbol table info available.
#12 0x0008dacf in mail_index_transaction_commit_full ()
No symbol table info available.
#13 0x0008db18 in mail_index_transaction_commit ()
No symbol table info available.
#14 0x00096434 in mail_index_sync_commit ()
No symbol table info available.
#15 0x00066f02 in maildir_sync_index_finish ()
No symbol table info available.
#16 0x00068376 in maildir_sync_context ()
No symbol table info available.
#17 0x000686f2 in maildir_storage_sync_init ()
No symbol table info available.
#18 0x000295fd in mailbox_sync ()
No symbol table info available.
#19 0x7833 in cmd_select_full ()
No symbol table info available.
#20 0x7d49 in cmd_select ()
No symbol table info available.
#21 0x9f63 in client_command_input ()
No symbol table info available.
#22 0xa0c2 in client_command_input ()
No symbol table info available.
#23 0xa1d3 in client_handle_input ()
No symbol table info available.
#24 0x95bd in client_input ()
No symbol table info available.
#25 0x001171a7 in io_loop_call_io ()
No symbol table info available.
#26 0x00118492 in io_loop_handler_run ()
No symbol table info available.
#27 0x00117148 in io_loop_run ()
No symbol table info available.
#28 0x0010425a in master_service_run ()
No symbol table info available.
#29 0x00013c5a in main ()
No symbol table info available.
(gdb) quit
The program is running.  Exit anyway? (y or n) y
4604ds1-ynoe:~ root#



Re: [Dovecot] dovecot freezes when trying to get mail from maildir with mail

2012-02-21 Thread postmaster

From: Timo Sirainen t...@iki.fi
 #9  0x0007ba2f in mail_index_update_day_headers ()

So Dovecot calls localtime() and the process hangs. Something's badly
broken in your system related to timezone handling, nothing Dovecot can
do about it..

OK - Thanks for your help :-)

I'll take it up on the macports mailing lists then !

--
Bjarne D Mathiesen
København N ; Danmark ; Europa
++
denne besked er skrevet i et totalt M$-frit miljø
MacOS X 10.7.3 Lion ; 2.8GHz Intel Core i7 ; 16GB 1067MHz DDR3


Re: [Dovecot] passwd-file - lower limit for username?

2012-02-21 Thread Oon-Ee Ng
On Tue, Feb 21, 2012 at 5:58 PM, Timo Sirainen t...@iki.fi wrote:
 On Tue, 2012-02-21 at 17:33 +0800, Oon-Ee Ng wrote:
 Hi, previously I could use the following in my passwd-file:-

 AB:{plain}password

 I'm not sure when it stopped working, its not an account I access with
 any regularity, but today it would not work (2.1.0, 64-bit Arch
 Linux), with an 'unknown user' error in the logs. Is there some new
 lower limit for username length?

 Oh, I think it's because AB is uppercased. Either lowercase it ab or
 set this to empty:

 auth_username_format =

 I hadn't thought of this being a problem before.. Hm. Perhaps it's not a
 common problem, so I'll just change the Upgrading text to mention this
 possibility also..

Yes, its probably the upper-casing. No problem, its just that it used
to work and now doesn't, I've already changed the username (to
something longer which isn't all uppercase) anyway. Probably SHOULD be
mentioned briefly somewhere.

Thanks much.


Re: [Dovecot] dsync, hard-links and refcounts

2012-02-21 Thread Christoph Bußenius

Hi,

On 15.02.2012 04:46, Timo Sirainen wrote:


It worked when the GUID already existed somewhere in destination, but not if it 
was added only during the same session. The attached patch fixes it. I'll 
commit it to v2.1 hg after I'll release v2.1.0..


sorry to bother you again, but I think there is a problem with this patch:

If a maildir contains several copies of the same message all in the same 
folder, dsync will not deduplicate them.


While IMAP cannot directly create copies of a message in the same 
folder, it does still happen if you copy (or move) a message back and 
forth between two folders.


Cheers,
Christoph

--
Christoph Bußenius
Rechnerbetriebsgruppe der Fakultäten Informatik und Mathematik
Technische Universität München
+49 89-289-18519  Raum 00.05.055  Boltzmannstr. 3  Garching


Re: [Dovecot] dsync, hard-links and refcounts

2012-02-21 Thread Timo Sirainen
On Tue, 2012-02-21 at 11:23 +0100, Christoph Bußenius wrote:
 Hi,
 
 On 15.02.2012 04:46, Timo Sirainen wrote:
 
  It worked when the GUID already existed somewhere in destination, but not 
  if it was added only during the same session. The attached patch fixes it. 
  I'll commit it to v2.1 hg after I'll release v2.1.0..
 
 sorry to bother you again, but I think there is a problem with this patch:
 
 If a maildir contains several copies of the same message all in the same 
 folder, dsync will not deduplicate them.

Correct. I nearly finished implementing this also, but then I thought it
just makes the code unnecessarily complex for no good reason.

 While IMAP cannot directly create copies of a message in the same 
 folder, 

It can: SELECT INBOX, COPY 1 INBOX

 it does still happen if you copy (or move) a message back and 
 forth between two folders.

Is it common enough to be an actual problem?



Re: [Dovecot] Recalculate quota when quota=dict ?

2012-02-21 Thread Jacek Osiecki

W dniu 2012-02-21 02:13, Timo Sirainen napisał(a):

On 17.2.2012, at 11.51, jos...@hybrid.pl wrote:

By the way: what might have caused such a warning?
r...@mail2.hybrid.pl /tmp/transfer doveadm quota recalc -u 
jos...@hybrid.pl
doveadm(jos...@hybrid.pl): Warning: Created dotlock file's timestamp 
is different than current time (1329464622 vs 1329464672): 
/var/mail/mail/hybrid.pl/joshua/.mailing.ekg/dovecot-uidlist


Does it keep happening? Is this a local filesystem or NFS? Shouldn't
happen unless remote storage server's clock and local server's clock
aren't synced.


OK, that's the case - /var/mail is stored on remote NFS.
I'll check if I have time properly set, but if so - I'll probably have
to live with it (I have no influence on NFS server).

Greetings,
--
Jacek Osiecki


Re: [Dovecot] dsync, hard-links and refcounts

2012-02-21 Thread Christoph Bußenius

On 21.02.2012 12:04, Timo Sirainen wrote:

On Tue, 2012-02-21 at 11:23 +0100, Christoph Bußenius wrote:

Hi,

On 15.02.2012 04:46, Timo Sirainen wrote:


It worked when the GUID already existed somewhere in destination, but not if it 
was added only during the same session. The attached patch fixes it. I'll 
commit it to v2.1 hg after I'll release v2.1.0..


sorry to bother you again, but I think there is a problem with this patch:

If a maildir contains several copies of the same message all in the same
folder, dsync will not deduplicate them.


Correct. I nearly finished implementing this also, but then I thought it
just makes the code unnecessarily complex for no good reason.


While IMAP cannot directly create copies of a message in the same
folder,


It can: SELECT INBOX, COPY 1 INBOX


Oh, mea culpa :)




it does still happen if you copy (or move) a message back and
forth between two folders.


Is it common enough to be an actual problem?


Actually we have some mailboxes with massively duplicated messages in 
the same folder.  Of course I cannot tell how common it is in general. 
I could imagine that some people routinely copy all INBOX messages into 
archive folders and do not check whether the archive already contains 
these messages.


Apart from the waste of space, I was wondering:
Is it okay for an mdbox to have several duplicate instances of a message 
with the same GUID?  Might some kind of corruption arise from this?


Cheers,
Christoph

--
Christoph Bußenius
Rechnerbetriebsgruppe der Fakultäten Informatik und Mathematik
Technische Universität München
+49 89-289-18519  Raum 00.05.055  Boltzmannstr. 3  Garching


Re: [Dovecot] dsync, hard-links and refcounts

2012-02-21 Thread Timo Sirainen
On 21.2.2012, at 13.55, Christoph Bußenius wrote:

 Apart from the waste of space, I was wondering:
 Is it okay for an mdbox to have several duplicate instances of a message with 
 the same GUID?  Might some kind of corruption arise from this?

No corruption. And they might even become deduplicated if you do doveadm 
force-resync + purge.



Re: [Dovecot] Questions still about a uw-imap - dovecot conversion

2012-02-21 Thread Steve Campbell



On 2/20/2012 4:58 PM, Jim Lawson wrote:

On 2/20/12 3:36 PM, Steve Campbell wrote:


Thanks for that input. I still think I'm missing something since I 
too used the compatibility link that you pointed to. Only thing is 
that proceeding those namespaces, I used the first example of:


namespace {
  type = private
  separator = /
  prefix = #mbox/
  location = mbox:~/mail:INBOX=/var/mail/%u
  inbox = yes
  hidden = yes
  list = no
}
namespace {
  type = private
  separator = /
  prefix =
  location = maildir:~/mail
}

This seemed to get the ball rolling so that users could access 
anything at all. I still have the problem of client imap folders 
being different from webmail imap folders. That's pretty much why I'm 
thinking of using mbox as the INBOX and all other imap folders in the 
~/mail directory in maildir format.
You ought to be able to get webmail to see the same folders as the 
imap clients.




Should I remove the first two namespaces, in your opinion? Right now, 
the whole thing is kind of fragile.




From your added namespaces, it looks like you are trying to support 
mbox and maildir and ~/mail at the same time.  Do you already have 
maildir folders to support?  If not, I would try to get things working 
well with mbox first before I started a conversion to maildir.  I 
would also ask why you're thinking of moving to maildir.  Yes, there 
are caching benefits, but when you add the Dovecot indexing on top of 
mbox, it's pretty much a wash.  If you are using file-level backup, 
rather than some sort of snapshot technology, maildir will be much, 
much slower to back up.  Your system will spend all its time walking 
directories, opening and closing files. If you don't have many users 
to worry about it might be OK, but make sure it's worth it.  A lot of 
sites went to maildir in the 1998-2004 era and have regretted the 
decision as their systems get overloaded with files and they can't 
back them up.


No, I've got all mbox formats. Previous posts probably suggested that I 
wanted to move to maildir, but all of the replies I've received have 
convinced me that I do not want to do that. The folders in ~ and ~/mail 
are mboxes, so I need to see what damage I've done with a maildir 
namespace. The maildir reference could be part of the problems I'm seeing.


I'm still not sure whether I should be seeing .subscriptions or 
mail/subscriptions anywhere and whether Dovecot will use the 
.mailboxlist that exists. The wiki suggests that I need to recompile 
Dovecot to continue using .mailboxlist. This is something I don't want 
to do.


Horde/Imp updates are probably out of the question until I can get a 
server to install the upgrade on.
For Dovecot and IMP both, you should set up an alternate server to 
test out your config changes on before you put them into production.  
If you are running on a bare metal single server, set up another 
Dovecot instance on a different set of ports (I commonly use 20143 
(imap), 20993 (imaps), 20110 (pop3)) which you can fiddle with 
freely.  Once you are satisfied with the result in your various 
clients, put it into production.  You can do the same with Horde/IMP 
by putting an installation in a different location on your webserver.


Jim
I've got a second server that is totally independent of the one I'm 
messing with. I've done the horde/imp alternate port/config. It all went 
well. The downside is that I didn't realize those secondary folders in 
~/mail were being hidden. Kinda late to switch back, but the secondary 
server supports another domain, so I have it to test with.


Again, the damage I've caused to those secondary imap folders needs to 
be determined to get this fixed properly. Most users are seing the 
secondary folders and can use them. It's just those users who have 
folders in ~ that are seeing problems as far as I can tell.


Thanks

steve







Re: [Dovecot] Questions still about a uw-imap - dovecot conversion

2012-02-21 Thread Steve Campbell



On 2/20/2012 7:25 PM, Joseph Tam wrote:


Steve Campbell wrote:

The more I read about all of this, the more I'm thinking about moving 
to maildir format. My switchover this weekend is full of holes due to 
the way user's imap folders were laid out. Some had folders in their 
home directory and others might have folders in their /home/mail 
directory.


I had the same problem, and handled it more or less the same way as
the later poster (Jim Lawson jtl+dove...@uvm.edu) handled his site.
I retained use of mbox format as well.

I planned the migration like this

- scan user home directories for mailboxes (especially Trash,
Drafts, Sent, etc.) looking for From  as the first 5 bytes.

This was piped into a script that Email'd users about the changes
that was going to happen, what they would expect to see, and a
FAQ on how to set up a mail client correctly.


Here, I'm not sure what should be done. The users with the secondary 
folders that are not in ~/mail can't seem to get the client configured.


- during the cutover,

- mailboxes left on the home directory were moved to
~/mail, or renamed (e.g. Sent - 00Sent to avoid
name collision for users that had a mixture of correct
and blank prefixes.

I've tried this and modified the .mailboxlist, but I'm thinking Dovecot 
is ignoring this and I'm not sure what it's looking at to determine the 
imap folders.

(I think I deleted Trash, Junk, etc. anyways).

- Namespace aliasing was used so that prefixes , mail,
~/mail, and whatever darn fool settings my users used,
would map to the same directory.

- .subscription files were moved into the mail folder
(don't have to edit prefixes since the aliasing
will take care of that).


And when would these .subscription files be created? The first time the 
folders would be accessed, or when? This seems to be part of my fix that 
I'm getting lost on.


- depending on what POP3 client you used (I used
qpopper), you may need to configure

pop3_reuse_xuidl = yes

to avoid a massive re-downloading from POP3 clients
after cutover.

- the mail clients I control centrally (e.g. webmail,
public server mail clients, etc). shouldn't need updating
since they ought to have been set up properly in the first
place.

- after cutover, a second notification was sent for users that
didn't move their mailboxes the first time around, and was it
done for them during migration.

And of course, test like crazy and watch the logs like a hawk.

Other gotcha's:

- your setup is fairly close to mine, so you may also run into
the problem of user having mailboxes with group ownership that
users are not part of (for example, group mail for INBOX set
by your LDA or personal mailboxes with groups the user is no
longer a member of) *and* with group permission not mode=0.


Sendmail sort of requires the mail group, does it not? I'll take a 
look and see if all users are part of this or not. A crazy solution 
would be in order here?


You'll have to treat these (set mode=0, or change the group to
something the user is part of), or the dovecot index creation
will fail and they won't be able to access their mailboxes.

I think my migration went pretty smoothly.  Less than a handful of 
wazzup'

Email problem reports.


Except for those users with the different folder locations, it seems 
that all is going pretty well. Maybe they're just not notifying me yet, 
though.


Joseph Tam jtam.h...@gmail.com



Thanks
steve



Re: [Dovecot] Questions still about a uw-imap - dovecot conversion

2012-02-21 Thread Timo Sirainen
On 21.2.2012, at 14.19, Steve Campbell wrote:

 I'm still not sure whether I should be seeing .subscriptions or 
 mail/subscriptions anywhere and whether Dovecot will use the .mailboxlist 
 that exists. The wiki suggests that I need to recompile Dovecot to continue 
 using .mailboxlist. This is something I don't want to do.

With new enough Dovecot version (2.0, maybe 1.2) you can add 
:SUBSCRIPTIONS=.mailboxlist to mail_location.



[Dovecot] Where did the message end up?

2012-02-21 Thread Dag Nygren

Hi all!

Just migrated out UW-IMAP installation to Dovecot and so far so good.

Had small teething problems though which left me with one question.

The scenario was that I have a set of public folders that are only needed by 
the main company group (gid = company). Then there are some external users as 
well, not directly in the company group.

I set up the public mailbox with permissions 660, owner me and group 
company. That is the externals didn't have access to the mail folder.

At this point all looked fine, but later there was an email to an external 
person and this was passed on to deliver by procmail. Procmail logs says all 
was OK, but there is an entry about the public folder not accessible in the 
Dovecot log. The email is nowhere to be found.

My question:

Where is the delivered email?

Best
Dag


Re: [Dovecot] Some trouble with dovecot 2.1 and SASL

2012-02-21 Thread Alessio Cecchi

Il 21/02/2012 10:16, Timo Sirainen ha scritto:

On Sun, 2012-02-19 at 11:39 +0100, Alessio Cecchi wrote:


I'm testing dovecot 2.1.0, on Debian 6.0, with passdb driver = imap
and SASL for postfix's SMTP-Auth, like explained ih this howto:
http://wiki2.dovecot.org/HowTo/PostfixAndDovecotSASL

Authentications over IMAP (local dovecot -  remote dovecot) works fine,
and it's great. Also smtp-auth on postfix (over SASL) works fine. But in
the dovecot log I found this error:


Feb 19 11:11:42 demo-vpop dovecot: auth: Fatal: master: service(auth):
child 30506 killed with signal 11 (core dumps disabled)


http://hg.dovecot.org/dovecot-2.1/rev/a6a58512f0df should fix this.




Patch applied, after rebuilding all works fine.

Thanks Timo
--
Alessio Cecchi is:
@ ILS - http://www.linux.it/~alessice/
on LinkedIn - http://www.linkedin.com/in/alessice
Assistenza Sistemi GNU/Linux - http://www.cecchi.biz/
@ PLUG - ex-Presidente, adesso senatore a vita, http://www.prato.linux.it
@ LOLUG - Socio http://www.lolug.net


Re: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend

2012-02-21 Thread Ed W

On 13/02/2012 19:43, Jan-Frode Myklebust wrote:

On Mon, Feb 13, 2012 at 11:08:48AM -0800, Mark Moseley wrote:

Out of curiosity, are you running dovecot locally on those webmail
servers as well, or is it talking to remote dovecot servers?

The webmail servers are talking with dovecot director servers which in
turn are talking with the backend dovecot servers. Each service running
on different servers.

Webmail-servers -  director-servers -  backend-servers



I think the original question was still sensible.  In your case it seems 
like the ping times are identical between:

webmail - imap-proxy
webmail - imap server

I think your results show that a proxy has little (or negative) benefit 
in this situation, but it seems feasible that a proxy could eliminate 
several RTT trips in the event that the proxy is closer than the imap 
server?  This might happen if say the imap server is in a different 
datacenter (webmail on an office server machine?)


I'm also pleased to see that there is little negative cost in using a 
proxy... I recently added imap-proxy to our webmail setup because I 
wanted to log last login + logout times.  I haven't quite figured out 
how to best log logout time (Timo, any chance of a post logout script? 
Or perhaps it's possible with the current login scripting?).  However, 
using imap-proxy has the benefit of clustering logins a little and 
this makes log files a little easier to understand in the face of users 
with desktop mail clients plus webmail users.  Possibly this idea useful 
to someone else...


Thanks for measuring this!

Ed W


[Dovecot] Managesieve AUTHENTICATE with IDN using UTF-8 or encode into ACE?

2012-02-21 Thread Christian Rohmann
Hey dovecot-users,

clients that speak telnet based protocols like http, smtp usually encode
IDN (International Domain Names) containing UTF-8 characters into ACE
(ASCII Compatible Encoding) punycode.

I am wondering what the case was with managesieve (RFC 5804). For
example if I was using a username like user@äüö.example.com with the
AUTHENTICATE command. Would that need to be encoded into ACE or be left
in UTF-8? Reading section 2.1. AUTHENTICATE Command or 4. Formal
Syntax suggests that quoted or literal UTF-8 is correct.



Thanks


Christian


Re: [Dovecot] Managesieve AUTHENTICATE with IDN using UTF-8 or encode into ACE?

2012-02-21 Thread Florian Zeitz
Am 21.02.2012 16:05, schrieb Christian Rohmann:
 Hey dovecot-users,
 
 clients that speak telnet based protocols like http, smtp usually encode
 IDN (International Domain Names) containing UTF-8 characters into ACE
 (ASCII Compatible Encoding) punycode.
 
 I am wondering what the case was with managesieve (RFC 5804). For
 example if I was using a username like user@äüö.example.com with the
 AUTHENTICATE command. Would that need to be encoded into ACE or be left
 in UTF-8? Reading section 2.1. AUTHENTICATE Command or 4. Formal
 Syntax suggests that quoted or literal UTF-8 is correct.

Hy,
what you're asking for is the correct format of the simple username.
This is in fact deployment specific. There is in general no requirement
to include the domain at all. The only requirement for to fulfil is that
it must not contain any codepoints prohibited by the SASLPrep stringprep
profile.
As such both the ACE and the IDN variant are valid. Which one to choose,
or whether to use only user depends on local policies.
There might be some implementation specifics in dovecot/pigeonhole that
I'm not aware of though.

Regards,
Florian


Re: [Dovecot] dsync, hard-links and refcounts

2012-02-21 Thread Christoph Bußenius

On 21.02.2012 13:15, Timo Sirainen wrote:


And they might even become deduplicated if you do doveadm force-resync + purge.



I hadn't tried that yet.  Thanks for the hint, this is probably all we need.

Cheers,
Christoph

--
Christoph Bußenius
Rechnerbetriebsgruppe der Fakultäten Informatik und Mathematik
Technische Universität München
+49 89-289-18519  Raum 00.05.055  Boltzmannstr. 3  Garching


Re: [Dovecot] Failing: doveadm sync --remote host-- dsync mirror

2012-02-21 Thread Michael Grimm
Hi --

On 21.02.2012, at 02:02, Timo Sirainen wrote:
 On 18.2.2012, at 20.26, Michael Grimm wrote:

 doveadm(test): Fatal: All your namespaces have a location setting. It should 
 be empty (default mail_location) in the namespace to be converted.
 
 Fixed: http://hg.dovecot.org/dovecot-2.1/rev/488fe9d417eb

Hmm, now I cannot clients any longer:

dovecot: master: Dovecot v20120221 (7d0d76df719f+) starting up
dovecot: imap-login: Login: user=test, method=PLAIN, rip=xxx.xxx.xxx.xxx, 
lip=yyy.yyy.yyy.yyy, mpid=65444, TLS
dovecot: imap(test): Error: user test: Initialization failed: mail_location not 
set and autodetection failed: Mail storage autodetection failed with 
home=/var/mail/.homedirs/test
dovecot: imap(test): Error: Invalid user settings. Refer to server log for more 
information.
[... repeated for every user clients are trying to connect ...]

I switched back immediately (production server), thus I didn't test if dsync 
might run now.

Here's the relevant part of my config:

mail_home = /var/mail/.homedirs/%n
mail_location = mdbox:/var/mail/.maildirs/%n

User's individual mail storage formats are fetched from userdb (sqlite3), which 
normally are mdbox, except for test which is sdbox.


Any idea what to test next?

Thanks and regards,
Michael



Re: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend

2012-02-21 Thread Timo Sirainen
On 21.2.2012, at 16.33, Ed W wrote:

 I'm also pleased to see that there is little negative cost in using a 
 proxy... I recently added imap-proxy to our webmail setup because I wanted to 
 log last login + logout times.  I haven't quite figured out how to best log 
 logout time (Timo, any chance of a post logout script? Or perhaps it's 
 possible with the current login scripting?).

You could of course grep the logs, but other than that you'd need to write a 
Dovecot plugin. Luckily it's really simple to write a plugin. Basically:

void postlogout_init(struct module *module) { }
void postlogout_deinit(void) {
  system(/usr/local/bin/dovecot-postlogout.sh);
} 

Add a few missing #includes and compile and enable for imap/pop3 and that 
should be it.



Re: [Dovecot] Failing: doveadm sync --remote host-- dsync mirror

2012-02-21 Thread Timo Sirainen
Hi,

On 21.2.2012, at 19.29, Michael Grimm wrote:

 doveadm(test): Fatal: All your namespaces have a location setting. It 
 should be empty (default mail_location) in the namespace to be converted.
 
 Fixed: http://hg.dovecot.org/dovecot-2.1/rev/488fe9d417eb
 
 Hmm, now I cannot clients any longer:
 
 dovecot: imap(test): Error: user test: Initialization failed: mail_location 
 not set and autodetection failed: Mail storage autodetection failed with 
 home=/var/mail/.homedirs/test

Sorry, that part of the code is a bit weird. 
http://hg.dovecot.org/dovecot-2.1/rev/3007994141b6 is another attempt at this.



Re: [Dovecot] Failing: doveadm sync --remote host-- dsync mirror

2012-02-21 Thread Michael Grimm
Hi --

On 21.02.2012, at 21:52, Timo Sirainen wrote:
 On 21.2.2012, at 19.29, Michael Grimm wrote:

 doveadm(test): Fatal: All your namespaces have a location setting. It 
 should be empty (default mail_location) in the namespace to be converted.
 
 Fixed: http://hg.dovecot.org/dovecot-2.1/rev/488fe9d417eb
 
 Hmm, now I cannot connect clients any longer:
 
 dovecot: imap(test): Error: user test: Initialization failed: mail_location 
 not set and autodetection failed: Mail storage autodetection failed with 
 home=/var/mail/.homedirs/test
 
 Sorry, that part of the code is a bit weird. 
 http://hg.dovecot.org/dovecot-2.1/rev/3007994141b6 is another attempt at this.

Thanks, that omits the first and second error, and I believe that you solved 
this issue.

Now, I can run my old command as used for 2.0.x and ...

vmail dsync -v -f -u test mirror ssh vm...@remote-host.tld dsync -v -f 
-u test

... a newer call like ...

vmail doveadm sync -f -u test ssh vm...@remote-host.tld dsync -v -f -u 
test

... without complaining about namespaces as listed above.


But, now I do end with a message like ...

vmail dsync -v -f -u test mirror ssh vm...@remote-host.tld dsync -v -f 
-u test
usage: dsync [-C alt char] [-m mailbox] [-u user] [-frRv]
 mirror local mail_location | [user@]host | remote dsync command
dsync-local(test): Error: read() from worker server failed: EOF

... irrespectively if I use the old dsync or the new doveadm sync.

Ok, this is still a mixed 2.0.18 --remote host-- 2.1.0 setup, but as long I 
didn't get your http://wiki2.dovecot.org/Upgrading/2.1 wrong, it *should* work 
that way.

Weird. I must have missed something. I'll back to 2.0.18 for the time being. 
Any hints are highly welcome ...

Thanks for your efforts, and thanks for dovecot!

Regards,
Michael



Re: [Dovecot] Failing: doveadm sync --remote host-- dsync mirror

2012-02-21 Thread Timo Sirainen
On 21.2.2012, at 23.48, Michael Grimm wrote:

 But, now I do end with a message like ...
 
   vmail dsync -v -f -u test mirror ssh vm...@remote-host.tld dsync -v -f 
 -u test
   usage: dsync [-C alt char] [-m mailbox] [-u user] [-frRv]
mirror local mail_location | [user@]host | remote dsync command
   dsync-local(test): Error: read() from worker server failed: EOF
 
 ... irrespectively if I use the old dsync or the new doveadm sync.
 
 Ok, this is still a mixed 2.0.18 --remote host-- 2.1.0 setup,

Probably intentional. Even if it didn't give that error message, it would fail 
because v2.0 and v2.1 uses a slightly different protocol (I thought about 
making it backwards compatible, but that seemed like too much work and too much 
ugliness to code).

 but as long I didn't get your http://wiki2.dovecot.org/Upgrading/2.1 wrong, 
 it *should* work that way.

Added it there.



Re: [Dovecot] Failing: doveadm sync --remote host-- dsync mirror

2012-02-21 Thread Michael Grimm
Hi --

On 21.02.2012, at 22:57, Timo Sirainen wrote:
 On 21.2.2012, at 23.48, Michael Grimm wrote:

 But, now I do end with a message like ...
 
  vmail dsync -v -f -u test mirror ssh vm...@remote-host.tld dsync -v -f 
 -u test
  usage: dsync [-C alt char] [-m mailbox] [-u user] [-frRv]
   mirror local mail_location | [user@]host | remote dsync command
  dsync-local(test): Error: read() from worker server failed: EOF
 
 ... irrespectively if I use the old dsync or the new doveadm sync.
 
 Ok, this is still a mixed 2.0.18 --remote host-- 2.1.0 setup,
 
 Probably intentional. Even if it didn't give that error message, it would 
 fail because v2.0 and v2.1 uses a slightly different protocol (I thought 
 about making it backwards compatible, but that seemed like too much work and 
 too much ugliness to code).

Ok. Then I will try both servers upgraded to 2.1 within the next days. Thanks 
for that valuable info that I really missed, sorry.

 but as long I didn't get your http://wiki2.dovecot.org/Upgrading/2.1 wrong, 
 it *should* work that way.
 
 Added it there.

Great.

And thanks again,
Michael




[Dovecot] duplicates with multiple To/CC and sieve redirect copy

2012-02-21 Thread Adam Szpakowski

Hi,
In one installation we are using sieve redirect :copy to create copy 
of the incoming email (local boss gets copy of its employees mails).
There is a problem when the incoming email has multiple To/CC entries 
with local users. Multiple copy/redirects are created (one for each 
employee) and there are multiple copies in boss account.

Each of this emails has the same Message-ID.

Is there a way to detect those duplicates and discard them when saving 
copies into boss account?


There was a discussion on the list a while ago (Feb 10/11, 2011) 
mentioning this problem, but with no clear solution.


We are using Dovecot 2.0.15. Mail is delivered via LMTP (MDA/postfix is 
on separate machine then MAA/dovecot).


--
Adam Szpakowski


Re: [Dovecot] duplicates with multiple To/CC and sieve redirect copy

2012-02-21 Thread Timo Sirainen
On 22.2.2012, at 0.46, Adam Szpakowski wrote:

 In one installation we are using sieve redirect :copy to create copy of the 
 incoming email (local boss gets copy of its employees mails).
 There is a problem when the incoming email has multiple To/CC entries with 
 local users. Multiple copy/redirects are created (one for each employee) and 
 there are multiple copies in boss account.

Could this instead be implemented by MTA as automatic bcc: boss of all incoming 
mails?



Re: [Dovecot] duplicates with multiple To/CC and sieve redirect copy

2012-02-21 Thread Adam Szpakowski

On 21.02.2012 23:57, Timo Sirainen wrote:

On 22.2.2012, at 0.46, Adam Szpakowski wrote:

In one installation we are using sieve redirect :copy to create copy of the 
incoming email (local boss gets copy of its employees mails).
There is a problem when the incoming email has multiple To/CC entries with local users. 
Multiple copy/redirects are created (one for each employee) and there are multiple copies 
in boss account.

Could this instead be implemented by MTA as automatic bcc: boss of all incoming 
mails?
Unfortunately no. The employee/boss structure is quite complicated. Some 
employees have multiple bosses, there are also some project emails 
when the structure is even more complicated.
We are using dedicated sieve_before script for each user based on %u 
variable.

--
Adam Szpakowski


Re: [Dovecot] duplicates with multiple To/CC and sieve redirect copy

2012-02-21 Thread Timo Sirainen
On 22.2.2012, at 1.02, Adam Szpakowski wrote:

 On 21.02.2012 23:57, Timo Sirainen wrote:
 On 22.2.2012, at 0.46, Adam Szpakowski wrote:
 In one installation we are using sieve redirect :copy to create copy of 
 the incoming email (local boss gets copy of its employees mails).
 There is a problem when the incoming email has multiple To/CC entries with 
 local users. Multiple copy/redirects are created (one for each employee) 
 and there are multiple copies in boss account.
 Could this instead be implemented by MTA as automatic bcc: boss of all 
 incoming mails?
 Unfortunately no. The employee/boss structure is quite complicated. Some 
 employees have multiple bosses, there are also some project emails when 
 the structure is even more complicated.
 We are using dedicated sieve_before script for each user based on %u variable.

Well, it would be possible to build a doveadm script that deletes the 
duplicates after delivery, but currently there's no implementation to avoid 
delivering duplicate Message-IDs in the first place.

I don't really like such a Message-ID-based deduplication feature enabled by 
default, but something like this could be nice:

fileinto :copy :x-deduplicate boss;

Anyway, probably not going to be implemented anytime soon.



Re: [Dovecot] duplicates with multiple To/CC and sieve redirect copy

2012-02-21 Thread Adam Szpakowski

On 22.02.2012 00:09, Timo Sirainen wrote:

On 22.2.2012, at 1.02, Adam Szpakowski wrote:

On 21.02.2012 23:57, Timo Sirainen wrote:

On 22.2.2012, at 0.46, Adam Szpakowski wrote:

In one installation we are using sieve redirect :copy to create copy of the 
incoming email (local boss gets copy of its employees mails).
There is a problem when the incoming email has multiple To/CC entries with local users. 
Multiple copy/redirects are created (one for each employee) and there are multiple copies 
in boss account.

Could this instead be implemented by MTA as automatic bcc: boss of all incoming 
mails?

Unfortunately no. The employee/boss structure is quite complicated. Some employees have multiple 
bosses, there are also some project emails when the structure is even more 
complicated.
We are using dedicated sieve_before script for each user based on %u variable.

Well, it would be possible to build a doveadm script that deletes the 
duplicates after delivery, but currently there's no implementation to avoid 
delivering duplicate Message-IDs in the first place.

I don't really like such a Message-ID-based deduplication feature enabled by 
default, but something like this could be nice:

fileinto :copy :x-deduplicate boss;

Anyway, probably not going to be implemented anytime soon.

Maybe there is a way to use a procmail with something like this:

:0 Wh: msgid.lock
| formail -D 8192 .msgid.cache

But is there a safe way to use it together with sieve? Using Pigeonhole 
Sieve Pipe Plugin?


--
Adam Szpakowski


Re: [Dovecot] Where did the message end up?

2012-02-21 Thread Timo Sirainen
On 21.2.2012, at 14.56, Dag Nygren wrote:

 Just migrated out UW-IMAP installation to Dovecot and so far so good.
 
 Had small teething problems though which left me with one question.
 
 The scenario was that I have a set of public folders that are only needed by 
 the main company group (gid = company). Then there are some external users as 
 well, not directly in the company group.

So this is in a public namespace?

 I set up the public mailbox with permissions 660, owner me and group 
 company. That is the externals didn't have access to the mail folder.
 
 At this point all looked fine, but later there was an email to an external 
 person and this was passed on to deliver by procmail. Procmail logs says all 
 was OK, but there is an entry about the public folder not accessible in the 
 Dovecot log. The email is nowhere to be found.

Exact error message would be helpful. Was the error from deliver, or imap/pop3? 
If it was from deliver, it probably returned temporary failure.

 Where is the delivered email?

Have you checked what happens if deliver tempfails (e.g. make procmail call 
tempfail.sh script that just exits with 75), does procmail also correctly 
tempfail and tell MTA to retry the delivery later?

Re: [Dovecot] Questions still about a uw-imap - dovecot conversion

2012-02-21 Thread Joseph Tam


Steve Campbell campb...@cnpapers.com writes:


 This was piped into a script that Email'd users about the changes
 that was going to happen, what they would expect to see, and a
 FAQ on how to set up a mail client correctly.

Here, I'm not sure what should be done. The users with the secondary 
folders that are not in ~/mail can't seem to get the client configured.


Great artists steal.  Google around for a good set of instructions
on how to set up IMAP prefix's for various mail clients, and refer
your users to that.

I also referred them to our webmail (with the correct mail prefix) as a
reference to a list of mailboxes they ought to see.  If they don't match,
then that's a symptom of a bad prefix.

I've tried this and modified the .mailboxlist, but I'm thinking Dovecot 
is ignoring this and I'm not sure what it's looking at to determine the 
imap folders.


 - .subscription files were moved into the mail folder
 (don't have to edit prefixes since the aliasing
 will take care of that).

And when would these .subscription files be created? The first time the 
folders would be accessed, or when? This seems to be part of my fix that 
I'm getting lost on.


Sorry, as Timo intimated, dovecot uses .subscription, which can
be adjusted back to .mailboxlist, which uw-imapd uses.  You shouldn't
have to edit it if you set up namespace aliasing.  Just move it from
the top-level into your user's mail directory during cutover:

(if you use dovecot's default subscription filename)
mv $USER/.mailboxlist $USER/mail/.subscription

If your IMAP client is still ignoring subscriptions, it usually
means the client was configured that way.


 - your setup is fairly close to mine, so you may also run into
 the problem of user having mailboxes with group ownership that
 users are not part of (for example, group mail for INBOX set
 by your LDA or personal mailboxes with groups the user is no
 longer a member of) *and* with group permission not mode=0.

Sendmail sort of requires the mail group, does it not? I'll take a 
look and see if all users are part of this or not. A crazy solution 
would be in order here?


I think sendmail works fine if you set your mail spool mode=1777 (like
/tmp), but have each inbox mode=0600.  This setting will persist.

It's sendmail LDA mail.local that's the problem: it autocreates new
inboxes with permissions user:mail/660 and unless the user is in group
mail, chaos will ensue.

You can maybe do a cron script or some other hack, or better yet,
use dovecot LDA, which sets up the perms correctly (and keeps indexes
updated).  A one time chmod go-rwx during cutover is all that is
required then.

The log entry that points this problem out looks like

dovecot: imap(smith): Error: chown(/dc-cache/smith/.imap/xx, 
group=6(mail)) failed: Operation not permitted (egid=5678(goodgroup), group 
based on /home/smith/mail/xx - see http://wiki2.dovecot.org/Errors/ChgrpNoPerm)
dovecot: imap(smith): Error: mkdir(/dc-cache/smith/.imap/xx) failed: 
Not owner

Joseph Tam jtam.h...@gmail.com


Re: [Dovecot] Questions still about a uw-imap - dovecot conversion

2012-02-21 Thread Steve Campbell
Quoting Joseph Tam jtam.h...@gmail.com:

 
 Steve Campbell campb...@cnpapers.com writes:
 
   This was piped into a script that Email'd users about the changes
   that was going to happen, what they would expect to see, and a
   FAQ on how to set up a mail client correctly.
  
  Here, I'm not sure what should be done. The users with the secondary 
  folders that are not in ~/mail can't seem to get the client configured.
 
 Great artists steal.  Google around for a good set of instructions
 on how to set up IMAP prefix's for various mail clients, and refer
 your users to that.

This conversion to dovecot has been a real learning experience for me. The old
stuff that came with Centos 3 just worked out of the box. I never realized how
much could be modified. It was always there, I just never needed it.
 
 I also referred them to our webmail (with the correct mail prefix) as a
 reference to a list of mailboxes they ought to see.  If they don't match,
 then that's a symptom of a bad prefix.

Our web mail, Imp, seems to be doing fine now that all of the mboxes have been
moved to ~/mail.
 
  I've tried this and modified the .mailboxlist, but I'm thinking Dovecot 
  is ignoring this and I'm not sure what it's looking at to determine the 
  imap folders.
 
   - .subscription files were moved into the mail folder
   (don't have to edit prefixes since the aliasing
   will take care of that).
  
  And when would these .subscription files be created? The first time the 
  folders would be accessed, or when? This seems to be part of my fix that 
  I'm getting lost on.
 
 Sorry, as Timo intimated, dovecot uses .subscription, which can
 be adjusted back to .mailboxlist, which uw-imapd uses.  You shouldn't
 have to edit it if you set up namespace aliasing.  Just move it from
 the top-level into your user's mail directory during cutover:
 
   (if you use dovecot's default subscription filename)
   mv $USER/.mailboxlist $USER/mail/.subscription
 
 If your IMAP client is still ignoring subscriptions, it usually
 means the client was configured that way.
 
   - your setup is fairly close to mine, so you may also run into
   the problem of user having mailboxes with group ownership that
   users are not part of (for example, group mail for INBOX set
   by your LDA or personal mailboxes with groups the user is no
   longer a member of) *and* with group permission not mode=0.
  
  Sendmail sort of requires the mail group, does it not? I'll take a 
  look and see if all users are part of this or not. A crazy solution 
  would be in order here?
 
 I think sendmail works fine if you set your mail spool mode=1777 (like
 /tmp), but have each inbox mode=0600.  This setting will persist.
 
 It's sendmail LDA mail.local that's the problem: it autocreates new
 inboxes with permissions user:mail/660 and unless the user is in group
 mail, chaos will ensue.
 

So far, all things seem OK in this area. No complaints are being forwarded.
 You can maybe do a cron script or some other hack, or better yet,
 use dovecot LDA, which sets up the perms correctly (and keeps indexes
 updated).  A one time chmod go-rwx during cutover is all that is
 required then.
 
 The log entry that points this problem out looks like
 
   dovecot: imap(smith): Error: chown(/dc-cache/smith/.imap/xx, 
 group=6(mail))
 failed: Operation not permitted (egid=5678(goodgroup), group based on
 /home/smith/mail/xx - see http://wiki2.dovecot.org/Errors/ChgrpNoPerm)
   dovecot: imap(smith): Error: mkdir(/dc-cache/smith/.imap/xx) failed: Not
 owner
 
 Joseph Tam jtam.h...@gmail.com

For now, it looks like the only problem I'm seeing is that one user who uses
Outlook Express that has a pop account for one email account and an imap account
for another email account is having problems with the Sent folder. When she
sends an email using the imap account, the Sent copy goes into the pop
account's sent item folder. When using webmail, the sent copy goes into the
sent-mail file as it's supposed to. I'm guessing for now that that google you
suggested might point me to a fix for that.

Thanks all for all the great help.

Shared accounts are next, so wish me luck.

steve


 




-
This mail sent through IMP: http://horde.org/imp/


Re: [Dovecot] Where did the message end up?

2012-02-21 Thread Dag Nygren
onsdag 22 februari 2012 01:32:56 skrev  Timo Sirainen:
 On 21.2.2012, at 14.56, Dag Nygren wrote:
  Just migrated out UW-IMAP installation to Dovecot and so far so good.
  
  Had small teething problems though which left me with one question.
  
  The scenario was that I have a set of public folders that are only
  needed by the main company group (gid = company). Then there are some
  external users as well, not directly in the company group.
 
 So this is in a public namespace?

Yep, it was.
The definition:

namespace {
  type = public
  separator = /
  prefix = MailingLists/
  location = mdbox:/usr/local/var/mail/MailingLists
  inbox = no
  subscriptions = no  # v1.1+
}

  I set up the public mailbox with permissions 660, owner me and group
  company. That is the externals didn't have access to the mail folder.
  
  At this point all looked fine, but later there was an email to an
  external person and this was passed on to deliver by procmail. Procmail
  logs says all was OK, but there is an entry about the public folder not
  accessible in the Dovecot log. The email is nowhere to be found.
 
 Exact error message would be helpful. Was the error from deliver, or
 imap/pop3? If it was from deliver, it probably returned temporary failure.

The log parts with final names and groups changed..

Deliver error:

Feb 21 10:02:24 dag dovecot: lda(extuser): Error: user extuser: Initialization 
failed: Namespace 'MailingLists/': 
stat(/usr/local/var/mail/MailingLists/mailboxes) failed: Permission denied 
(euid=2002(extuser) egid=2002(extgroup) missing +x perm: 
/usr/local/var/mail/MailingLists, euid is not dir owner)
Feb 21 10:02:24 dag dovecot: lda(extuser): Fatal: Invalid user settings. Refer 
to server log for more information.

Procmail log:

From mailing-1341698764...@mailpv.net  Tue Feb 21 10:02:24 2012
 Subject: =?CP1250?Q?P=E4=E4si=E4issuunnitelmat?=
  Folder: deliver -d extuser -m inbox56979

  Where is the delivered email?
 
 Have you checked what happens if deliver tempfails (e.g. make procmail call
 tempfail.sh script that just exits with 75), does procmail also correctly
 tempfail and tell MTA to retry the delivery later?

No, haven't done that. yet. And I don't think it will pass on the failure 
code. Procmail settings are all from the HOWTO:s though.

The .forward file looks like this:

|IFS=' '  exec /usr/bin/procmail -f- || exit 75 #extuser

And the applicable rule in .procmailrc:
:0 w
{
:0
 | $DELIVERYAGENT -d $LOGNAME -m inbox
}

Have to point out that this case wasn't important, but I wouldn't like the 
system to loose an important mail later.





Re: [Dovecot] Where did the message end up?

2012-02-21 Thread Timo Sirainen
On 22.2.2012, at 9.34, Dag Nygren wrote:

 Feb 21 10:02:24 dag dovecot: lda(extuser): Error: user extuser: 
 Initialization 
 failed: Namespace 'MailingLists/': 
 stat(/usr/local/var/mail/MailingLists/mailboxes) failed: Permission denied 
 (euid=2002(extuser) egid=2002(extgroup) missing +x perm: 
 /usr/local/var/mail/MailingLists, euid is not dir owner)
 Feb 21 10:02:24 dag dovecot: lda(extuser): Fatal: Invalid user settings. 
 Refer 
 to server log for more information.

Right, the mail wasn't delivered, LDA simply exited with tempfail. Actually 
v2.1.0 is the first release where you can easily configure Dovecot to simply 
skip the namespace when it doesn't have permission to access it:

+ Added ignore_on_failure setting for namespaces. If namespace
  initialization fails with this enabled (e.g. permission denied),
  the namespace is silently skipped for the user.

 Where is the delivered email?
 
 Have you checked what happens if deliver tempfails (e.g. make procmail call
 tempfail.sh script that just exits with 75), does procmail also correctly
 tempfail and tell MTA to retry the delivery later?
 
 No, haven't done that. yet. And I don't think it will pass on the failure 
 code. Procmail settings are all from the HOWTO:s though.
 
 The .forward file looks like this:
 
 |IFS=' '  exec /usr/bin/procmail -f- || exit 75 #extuser
 
 And the applicable rule in .procmailrc:
 :0 w
 {
:0
 | $DELIVERYAGENT -d $LOGNAME -m inbox
 }
 
 Have to point out that this case wasn't important, but I wouldn't like the 
 system to loose an important mail later.

You should definitely make sure that procmail won't lose any mails if Dovecot 
LDA returns tempfailure, since they do sometimes happen. You could also get rid 
of procmail completely by moving to sieve.