Hello,
As of revision 14388, the Chandler Mail Service has under gone a
significant architectural change
to support downloading of large Inboxes (12,000 plus messages). To
accomplish this the
Incoming Mail logic has been refactored in to two asynchronous threads
the Twisted Mail Client thread and the Mail Worker thread. The Twisted
Mail Client thread handles all downloading and pre-processing of mail
(building the body text, finding eimml / ics attachments etc). The
Mail Worker thread is responsible for taking the pre-processed mail off
of a queue and
assigning the values to a MailStamped Item. There are now only two views
in play.
The Twisted Mail Client view shared by all downloading clients and the
Mail Worker view used to convert pre-processed mail from all Clients.
With this
check in performance has greatly increased, timeouts have been
minimized, and
a pesky bus error that occurs when too many Twisted deferreds are on the
Python
stack has been resolved.
Taken from the r14388 commit message:
======================================
Performance Performance Performance! Complete refactoring of Incoming
Mail Logic
to a Mail Worker thread with one view and all clients and Mail Service
sharing a second view.
The Mail Service can now download 12,000 messages with No timeouts and
Bus Errors.
There are still performance issues in the UI and Repo layer to tackle
related to merging,
refresh, and observer calls. But much improvement has been already made.
Fixed the Bus Error related to too many Twisted deferreds over flowing
the stack as well as added dynamic
commit calculation and throttling. Improved POP performance 2x when the
POP3 Server does not support the TOP command (gmail).Added IMAP
Body.PEEK commands to prevent mail on the IMAP server from being marked
as seen on download. Added Blocking vs. Non-Blocking logic.
The Incoming Mail Clients can either wait till the Mail Worker has
processed the last commit request or continue downloading mail while the
Mail Worker is converting Email Objects to Items.
Added Preview specific Message Parsing and Conversion to Item methods.
These are intended to maximize RFC2882 text to MailStamped Items
performance. Improved status bar messages when searching for Chandler
Headers and when downloading and committing mail. In addition fixed a
number of open Preview mail bugs.
Please not the Chandler Schema has changed as of rev 14388.
Enjoy,
Brian
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Open Source Applications Foundation "chandler-dev" mailing list
http://lists.osafoundation.org/mailman/listinfo/chandler-dev
