[
https://issues.apache.org/jira/browse/JAMES-3769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17537395#comment-17537395
]
Benoit Tellier commented on JAMES-3769:
---------------------------------------
I attached a screenshot of a slow IMAP search caputred thanks to glowroot. We
can see that for this very mailbox it takes 60+ seconds on a recurring basis.
Investigations linked this to Samsung Email application.
Corresponding log:
{code:java}
{
"_index": "fluentbit-james-2020.10.12",
"_type": "docker",
"_id": "QVmVGnUBdqSpI3q1OW1d",
"_score": 1,
"_source": {
"@timestamp": "2020-10-12T02:13:06.065Z",
"timestamp": "2020-10-12T02:13:06.065Z",
"level": "DEBUG",
"thread": "imapserver-executor-14160",
"mdc": {
"protocol": "IMAP",
"useUid": "true",
"selectedMailbox": "xxxxxxxxxx-ff03-11ea-b557-d14871188c2e",
"ip": "59.153.253.18",
"host": "59.153.253.18",
"action": "SEARCH",
"searchOperation": "SearchOperation{key=SearchKey{type=TYPE_SEQUENCE_SET,
date=null, size=0, value=null, seconds=-1, modSeq=-1, uids=null,
sequences=[IdRange ( 153416->153440 )], keys=Optional.empty}, options=[]}",
"sessionId": "SID-eogriuzkclic",
"user": "Optional[x...@linagora.com]"
},
"logger": "org.apache.james.imap.processor.base.AbstractChainedProcessor",
"message": "Processing
SearchRequest{operation=SearchOperation{key=SearchKey{type=TYPE_SEQUENCE_SET,
date=null, size=0, value=null, seconds=-1, modSeq=-1, uids=null,
sequences=[IdRange ( 153416->153440 )], keys=Optional.empty}, options=[]},
useUids=true}",
"context": "default"
},
"fields": {
"@timestamp": [
"2020-10-12T02:13:06.065Z"
],
"timestamp": [
"2020-10-12T02:13:06.065Z"
]
}
}
{code}
> ElasticSearch search overrides
> ------------------------------
>
> Key: JAMES-3769
> URL: https://issues.apache.org/jira/browse/JAMES-3769
> Project: James Server
> Issue Type: Improvement
> Components: elasticsearch
> Reporter: Benoit Tellier
> Priority: Major
> Attachments: 74cec3c3-305a-4dd9-a2bd-f54ff210ecfc.png
>
>
> h3. Context
> IMAP SEARCH requests can of course be used for user searches. But many
> applications (eg Samsung email client) also uses
> IMAP SEARCH for resynchronization.
> The use of ElasticSearch in the context of resynchronization is a problem as:
> - Resynchronization incurs many requests that ElasticSearch is not made to
> cope with.
> - This have a performance impact and some 60+ seconds can be spotted.
> - ElasticSearch indexing is asynchronous wich is not ideal for
> resynchronization
> - An ElasticSearch downtime would imply a full downtime for these clients.
>
> Those resynchronization queries would better be served by the metadata
> database
> h3. Decision
> Define an API, `SearchOverride` that can be called on matching search queries
> and thus offload ElasticSearch.
> Provide an extension mechanism for search overrides, where potentially custom
> implementations can be loaded from the
> configuration.
> Provide well targeted `SearchOverride`, implemented on top of `Cassandra`
> database.
> h3. Consequences
> We expect to :
> - Observe less IMAP SEARCH slow requests
> - ElasticSearch downtime should not impact IMAP clients using IMAP SEARCH
> like IMAP Samsung application
> - Thus also be able to support a higher IMAP workload as Cassandra is more
> suited to resynchronisation requests
> ## Sample IMAP requests
> {code:java}
> SearchOperation{key=SearchKey{type=TYPE_UID, date=null, size=0, value=null,
> seconds=-1, modSeq=-1, uids=[IdRange : TYPE: FROM UID:
> MessageUid{uid=1}:MessageUid{uid=9223372036854775807}], sequences=null,
> keys=Optional.empty}, options=[]}
> {code}
> {code:java}
> SearchOperation{key=SearchKey{type=TYPE_AND, date=null, size=0, value=null,
> seconds=-1, modSeq=-1, uids=null, sequences=null, keys=Optional[[
> SearchKey{type=TYPE_SEQUENCE_SET, date=null, size=0, value=null,
> seconds=-1, modSeq=-1, uids=null, sequences=[IdRange ( 1->9223372036854775807
> )], keys=Optional.empty},
> SearchKey{type=TYPE_DELETED, date=null, size=0, value=null, seconds=-1,
> modSeq=-1, uids=null, sequences=null, keys=Optional.empty}]]},
> options=[COUNT]}
> {code}
> {code:java}
> SearchOperation{key=SearchKey{type=TYPE_AND, date=null, size=0, value=null,
> seconds=-1, modSeq=-1, uids=null, sequences=null, keys=Optional[[
> SearchKey{type=TYPE_UID, date=null, size=0, value=null, seconds=-1,
> modSeq=-1, uids=[IdRange : TYPE: FROM UID:
> MessageUid{uid=1}:MessageUid{uid=9223372036854775807}], sequences=null,
> keys=Optional.empty},
> SearchKey{type=TYPE_UNSEEN, date=null, size=0, value=null, seconds=-1,
> modSeq=-1, uids=null, sequences=null, keys=Optional.empty}]]},
> options=[COUNT]}
> {code}
> {code:java}
> SearchOperation{key=SearchKey{type=TYPE_AND, date=null, size=0, value=null,
> seconds=-1, modSeq=-1, uids=null, sequences=null, keys=Optional[[
> SearchKey{type=TYPE_SEQUENCE_SET, date=null, size=0, value=null,
> seconds=-1, modSeq=-1, uids=null, sequences=[IdRange ( 1->9223372036854775807
> )], keys=Optional.empty},
> SearchKey{type=TYPE_NOT, date=null, size=0, value=null, seconds=-1,
> modSeq=-1, uids=null, sequences=null, keys=Optional[[
> SearchKey{type=TYPE_DELETED, date=null, size=0, value=null,
> seconds=-1, modSeq=-1, uids=null, sequences=null, keys=Optional.empty}]]}]]},
> options=[ALL]}
> {code}
> {code:java}
> SearchOperation{key=SearchKey{type=TYPE_ALL, date=null, size=0, value=null,
> seconds=-1, modSeq=-1, uids=null, sequences=null, keys=Optional.empty},
> options=[]}
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org
