The following issue has been ASSIGNED.
======================================================================
http://www.dbmail.org/mantis/view.php?id=619
======================================================================
Reported By: asianux-keith
Assigned To: paul
======================================================================
Project: DBMail
Issue ID: 619
Category: Database layer
Reproducibility: always
Severity: minor
Priority: normal
Status: assigned
target:
======================================================================
Date Submitted: 04-Jul-07 03:39 CEST
Last Modified: 15-Mar-08 10:29 CET
======================================================================
Summary: sqlite locking: "sqlite3_get_table failed" results
in msgs not being inserted into db, msgs bounced.
Description:
When sending a message from thunderbird or evolution to a server with
postfix and dbmail 2.2.5 and a sqlite(3) backend, the messages are often
not delivered because dbmail-smtp/lmtp cannot lock/write to the sqlite db.
Non-delivered msgs bounce.
In each test, a msg is crafted with three local TO: address (including
self).
1st test: (logfile sqlite-non-delivery-log.txt)
dbmail-lmtp, dbmail-imapd and dbmail-pop3d are all running.
The client sending the msg uses imap. One receiving client is imap, and
the other is pop. postfix uses dbmail-smtp to deliver.
2nd test: (logfile sqlite-lmtp-non-delivery-log.txt)
dbmail-lmtp, dbmail-imapd and dbmail-pop3d are all running.
The client sending the msg uses imap. One receiving client is imap, and
the other is pop. postfix uses dbmail-lmtp to deliver.
3rd test: (logfile sqlite-pop_only-non-delivery-log.txt)
only dbmail-pop3d is running.
The client sending the msg uses pop. Two receiving clients are imap.
postfix uses dbmail-smtp to deliver.
======================================================================
----------------------------------------------------------------------
paul - 04-Jul-07 10:32
----------------------------------------------------------------------
I've added a retry loop to the sqlite code to help deal with these
situation. Please try the attached patch.
Also: try to lower the concurrency settings in postfix's master.cf. Set
the width of the lmtp and pipe transports to 1.
----------------------------------------------------------------------
asianux-keith - 05-Jul-07 06:57
----------------------------------------------------------------------
The patch by itself does not resolve the problem, even with just three
processes battling for control.
The process which I think had the initial lock took less than two seconds
to complete, and the two other processes fought and it seems both lost to
each other.
I'd suggest building in some incremental, or random, delay with each loop
of the retry code. Something between 1/10 and 1 sec?
On the other hand, limiting the concurrency in postfix had a dramatic
effect, and allowed the delivery of all messages (even without the patch)
in a simple test. They only issue was that copying the message to the
Sent folder with the client using imap would 'sometimes' hang.
Combining the two did even better, until I stressed it by sending five
different messages (all with three local TO: addresses). Not all five
were delivered.
user1 (imap client) sent 2 msgs. user2 (imap client) sent 1 msg. root
(pop3) sent the last two msgs.
Oddly enough, user1 received only three msgs, and returned 1 bounce msg;
user 2 received four msgs, and root received all five.
There was at least one hang copying to Sent.
To main.cf, I added
dbmail-lmtp_destination_concurrency_limit = 1
dbmail-smtp_destination_concurrency_limit = 1
and to master.cf, I changed
dbmail-lmtp unix - - n - 1 lmtp -o
disable_dns_lookups=yes
dbmail-smtp unix - n n - 1 pipe
Issue History
Date Modified Username Field Change
======================================================================
04-Jul-07 03:39 asianux-keith New Issue
04-Jul-07 10:32 paul Note Added: 0002282
04-Jul-07 10:33 paul File Added:
0001-add-retry-loop-to-sqlite-db_query.txt
04-Jul-07 10:34 paul Assigned To => paul
04-Jul-07 10:34 paul Severity major => minor
04-Jul-07 10:34 paul Status new => acknowledged
05-Jul-07 06:57 asianux-keith Note Added: 0002284
05-Jul-07 06:59 asianux-keith File Added: sqlite-stress-5.txt.gz
15-Mar-08 10:29 paul Status acknowledged => assigned
======================================================================
_______________________________________________
Dbmail-dev mailing list
[email protected]
http://twister.fastxs.net/mailman/listinfo/dbmail-dev
