[
https://issues.apache.org/jira/browse/MAILBOX-296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16017214#comment-16017214
]
Tellier Benoit commented on MAILBOX-296:
----------------------------------------
The idea is:
- to batch EXPUNGE by packages of 100 to not overwhelm DB
- to get rid of the IN clause in CassandraMessageDAO, prefer using individual
parallel reads, also chunked to not overwhelm DB
- to prepare CassandraMessageDAO select query
I used https://github.com/linagora/gatling-imap/pull/22 to run Gatling metrics
and conduct testing:
| branch | 5% | 20%| 100% |
| -- | -- | -- | -- |
| master | 105/103 |660/218/X/195 | X/X
| this one| 94/1034|371/256/191/206| 598/550|
X means failure. Results are in ms.
About these stats:
- Order of run was strictly the same (20%, 5%, 5%, 20%, 20%, 20%, 100%, 100%)
for warm up reasons.
- We did not get much performance improvements, but we see now we can expunge
large mailboxes, what was before impossible.
Hence this is a huge win: it solves aims at solving both of the very common
Cassandra failures upon IMAP.
> Improve Expunges over Cassandra
> -------------------------------
>
> Key: MAILBOX-296
> URL: https://issues.apache.org/jira/browse/MAILBOX-296
> Project: James Mailbox
> Issue Type: Improvement
> Components: cassandra
> Reporter: Tellier Benoit
>
> Large expunges are today failing on Cassandra.
> And overall performance does not seem good.
> We should pay attention to:
> - Limiting the number of concurrent queries to avoid killing the server
> - Prepare select statements in MessageDAO
> - Not use IN clause in MessageDAO
> - Use limit on incoming IDs on MessageDAO reads
> - Better chaining of future
> We will validate performance gains with gatling-imap tests.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
