[Dovecot] error bad file number with compressed mbox files
Hello, can dsync convert from compressed mbox to compressed mdbox format? When I use compressed mbox files, either with gzip or with bzip2, I can read the mails as usual, but I find the following errors in dovecots log file: imap(userxy): Error: nfs_flush_fcntl: fcntl(/home/hrz/userxy/Mail/mymbox.gz, F_RDLCK) failed: Bad file number imap(userxy): Error: nfs_flush_fcntl: fcntl(/home/hrz/userxy/Mail/mymbox.bz2, F_RDLCK) failed: Bad file number These errors also appear when I use dsync to convert the compressed mbox to mdbox format on a second dovecot server: /opt/local/bin/dsync -v -u userxy backup mdbox:/sanpool/mail/home/hrz/userxy/mdbox dsync(userxy): Error: nfs_flush_fcntl: fcntl(/home/hrz/userxy/Mail/mymbox.gz, F_RDLCK) failed: Bad file number But now dovecot does not find the mails in the folder mymbox.gz on the second dovecot server in mdbox format! The relevant part of the dovcot configuration is: # 2.0.16: /opt/local/etc/dovecot/dovecot.conf # OS: SunOS 5.10 sun4v mail_fsync = always mail_location = mbox:~/Mail:INBOX=/var/mail/%u mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = mail_log notify zlib mmap_disable = yes Thank you, -- Jürgen Obermann Hochschulrechenzentrum der Justus-Liebig-Universität Gießen Heinrich-Buff-Ring 44 Tel. 0641-9913054
Re: [Dovecot] Dsync fails on second sync for folders with dot in the name
On 2012-01-01 2:59 PM, Jan-Frode Myklebust janfr...@tanso.net wrote: I'm in the processes of running our first dsync backup of all users (from maildir to mdbox on remote server), and one problem I'm hitting that dsync will work fine on first run for some users, and then reliably fail whenever I try a new run: $ sudo dsync -u janfr...@example.net backup ssh -q mailbac...@repo1.example.net dsync -u janfr...@example.net $ sudo dsync -u janfr...@example.net backup ssh -q mailbac...@repo1.example.net dsync -u janfr...@example.net dsync-remote(janfr...@example.net): Error: Can't delete mailbox directory INBOX/a: Mailbox has children, delete them first The problem here seems to be that this user has a maildir named .a.b. On the backup side I see this as a/b/. So dsync doesn't quite seem to agree with itself for how to handle folders with dot in the name. dovecot -n output? What are you using for the namespace hierarchy separator? http://wiki2.dovecot.org/Namespaces -- Best regards, Charles
Re: [Dovecot] Dsync fails on second sync for folders with dot in the name
On Mon, Jan 02, 2012 at 09:51:00AM -0500, Charles Marcus wrote:
dovecot -n output? What are you using for the namespace hierarchy separator?
I have the folder format default separator (maildir .), but still dovecot
creates
directories named .a.b.
On receiving dsync server:
=
$ dovecot -n
# 2.0.14: /etc/dovecot/dovecot.conf
mail_location = mdbox:~/mdbox
mail_plugins = zlib
mdbox_rotate_size = 5 M
passdb {
driver = static
}
plugin {
zlib_save = gz
zlib_save_level = 9
}
protocols =
service auth-worker {
user = $default_internal_user
}
service auth {
unix_listener auth-userdb {
mode = 0600
user = mailbackup
}
}
ssl = no
userdb {
args = home=/srv/mailbackup/%256Hu/%d/%n
driver = static
}
On POP/IMAP-server:
=
$ doveconf -n
# 2.0.14: /etc/dovecot/dovecot.conf
auth_cache_size = 100 M
auth_verbose = yes
auth_verbose_passwords = sha1
disable_plaintext_auth = no
login_trusted_networks = 192.168.0.0/16
mail_gid = 3000
mail_location = maildir:~/:INDEX=/indexes/%1u/%1.1u/%u
mail_plugins = quota zlib
mail_uid = 3000
maildir_stat_dirs = yes
maildir_very_dirty_syncs = yes
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy
include variables body enotify environment mailbox date
mmap_disable = yes
namespace {
inbox = yes
location =
prefix = INBOX.
type = private
}
passdb {
args = /etc/dovecot/dovecot-ldap.conf.ext
driver = ldap
}
plugin {
quota = maildir:UserQuota
sieve = /sieve/%1u/%1.1u/%u/.dovecot.sieve
sieve_dir = /sieve/%1u/%1.1u/%u
sieve_max_script_size = 1M
zlib_save = gz
zlib_save_level = 6
}
postmaster_address = postmas...@example.net
protocols = imap pop3 lmtp sieve
service auth-worker {
user = $default_internal_user
}
service auth {
client_limit = 4521
unix_listener auth-userdb {
group =
mode = 0600
user = atmail
}
}
service imap-login {
inet_listener imap {
address = *
port = 143
}
process_min_avail = 4
service_count = 0
vsz_limit = 1 G
}
service imap-postlogin {
executable = script-login /usr/local/sbin/imap-postlogin.sh
}
service imap {
executable = imap imap-postlogin
process_limit = 2048
}
service lmtp {
client_limit = 1
inet_listener lmtp {
address = *
port = 24
}
process_limit = 25
}
service managesieve-login {
inet_listener sieve {
address = *
port = 4190
}
service_count = 1
}
service pop3-login {
inet_listener pop3 {
address = *
port = 110
}
process_min_avail = 4
service_count = 0
vsz_limit = 1 G
}
service pop3-postlogin {
executable = script-login /usr/local/sbin/pop3-postlogin.sh
}
service pop3 {
executable = pop3 pop3-postlogin
process_limit = 2048
}
ssl = no
userdb {
args = /etc/dovecot/dovecot-ldap.conf.ext
driver = ldap
}
protocol lmtp {
mail_plugins = quota zlib sieve
}
protocol imap {
imap_client_workarounds = delay-newmail
mail_plugins = quota zlib imap_quota
}
protocol pop3 {
mail_plugins = quota zlib
pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
pop3_uidl_format = UID%u-%v
}
protocol sieve {
managesieve_logout_format = bytes=%i/%o
}
-jf
[Dovecot] Problem with huge IMAP Archive after Courier migration
I have a mail server running Debian 6.0 with Courier IMAP to store
project related mail.
Currently the maildir of the archive (one user) contains about 37GB of data.
Our staff is acessing the archive via Outlook 2007 where they drag their
Exchange inbox or sent files to it.
The problem with courier is that is sometimes mixes up headers with
message bodies, so I wanted to migrate to dovecot.
I tried this on my proxy running Debian 7.0 with some test data and this
worked fine (OK, spent some hours to get the config files done - Dovecot
without authentication).
Dovecot version here is 2.0.15.
Tried it with our productive system today, but got Dovecot 1.2.15
installed on Debian 6.0
Config files and parameters I took from my test system were not
compatible and I didn't get it to work.
So I forced to install the Debisn 7.0 packages with 2.0.15 and finally
got the server running, I also restarted the whole machine to empty caches.
But the problem I got was that in the huge folder hierarchy the
downloaded headers in the individual folders disappeared, some folders
showed a few very old messages, some none. Also some subfolders disappeared.
I checked this with Outlook and Thunderbird. The difference was, that
Thunderbird shows more messages (but not all) than Outlook in some
folders, but also none in some others. Outlook brought up a message in
some cases, that the connection timed out, although I set the timeout to
60s.
After being frustrated uninstalled dovecot, went back to Courier and
folder contents are displayed correctly again.
Anyone a clue what's wrong here?
Finally some config information:
proxy-server:~# dovecot -n
# 2.0.15: /etc/dovecot/dovecot.conf
# OS: Linux 3.1.0-1-686-pae i686 Debian wheezy/sid
auth_debug_passwords = yes
auth_mechanisms = plain login
disable_plaintext_auth = no
namespace {
inbox = yes
location =
prefix = INBOX.
separator = .
type = private
}
passdb {
driver = pam
}
plugin {
sieve = ~/.dovecot.sieve
sieve_dir = ~/sieve
}
protocols = imap
ssl = no
ssl_cert = /etc/ssl/certs/dovecot.pem
ssl_key = /etc/ssl/private/dovecot.pem
userdb {
driver = passwd
}
Re: [Dovecot] Problem with huge IMAP Archive after Courier migration
On 1/2/2012 10:17 AM, Preacher wrote: ... So I forced to install the Debisn 7.0 packages with 2.0.15 and finally got the server running, I also restarted the whole machine to empty caches. But the problem I got was that in the huge folder hierarchy the downloaded headers in the individual folders disappeared, some folders showed a few very old messages, some none. Also some subfolders disappeared. I checked this with Outlook and Thunderbird. The difference was, that Thunderbird shows more messages (but not all) than Outlook in some folders, but also none in some others. Outlook brought up a message in some cases, that the connection timed out, although I set the timeout to 60s. ... Anyone a clue what's wrong here? Absolutely. What's wrong is a lack of planning, self education, and patience on the part of the admin. Dovecot gets its speed from its indexes. How long do you think it takes Dovecot to index 37GB of maildir messages, many thousands per directory, hundreds of directories, millions of files total? Until those indexes are built you will not see a complete folder tree and all kinds of stuff will be missing. For your education: Dovecot indexes every message and these indexes are the key to its speed. Normally indexing occurs during delivery when using deliver or lmtp, so the index updates are small and incremental, keeping performance high. You tried to do this and expected Dovecot to instantly process it all: http://www.youtube.com/watch?v=THVz5aweqYU If you don't know, that's a coal train car being dumped. 100 tons of coal in a few seconds. Visuals are always good teaching tools. I think this drives the point home rather well. -- Stan
[Dovecot] Newbie: LDA Isn't Logging
Hi,
I'm a newbie having some trouble getting deliver to log anything. Related to
this, there are no return values unless the -d is missing. I'm using LDAP to
store virtual domain and user account information.
Test #1: /usr/lib/dovecot/deliver -e -f mpa...@yahoo.com -d
z...@mailswansong.dom bad.mail
Expected result: supposed to fail, there's no zed account via ldap lookup and
supposed to get a return code per the wiki at http://wiki2.dovecot.org/LDA.
Supposed to log too.
Actual result: nothing gets delivered, no return code, nothing is logged.
Test #2: /usr/lib/dovecot/deliver -f mpa...@yahoo.com -d d...@mailswansong.dom
good.mail
Expected result: deliver to dude and return 0.
Actual result: delivers, but no return code. Nothing logged.
The wiki is vague about the difficulties of getting deliver LDA to log, but I
thought I had it covered in my config. I even opened permissions up wide (777)
on my log files specified below. Nothing gets logged.
The ONLY thing changed in 15-lda.conf is as follows.
protocol lda {
# Space separated list of plugins to load (default is global mail_plugins).
#mail_plugins = $mail_plugins
log_path = /var/log/dovecot/lda.log
info_log_path = /var/log/dovecot/lda-info.log
service auth {
unix_listener auth-client {
mode = 0600
user = vmail
}
}
I'm running plain Debian Testing and used dovecot from Debian's repository.
The end-goal is to write a qpsmtpd queue plugin, but I need to figure out
what's the matter first.
Thanks in advance.
mpapet
