[jira] [Commented] (JAMES-3277) Don't read outbox mailbox for each message in SetMessagesUpdateProcessor
[
https://issues.apache.org/jira/browse/JAMES-3277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17213533#comment-17213533
]
René Cordier commented on JAMES-3277:
-
[https://github.com/linagora/james-project/pull/3889] contributes some
refactoring to fasten Setmessages::update flags
> Don't read outbox mailbox for each message in SetMessagesUpdateProcessor
>
>
> Key: JAMES-3277
> URL: https://issues.apache.org/jira/browse/JAMES-3277
> Project: James Server
> Issue Type: Bug
> Components: JMAP
>Affects Versions: 3.5.0
>Reporter: Rémi Kowalski
>Assignee: Antoine Duprat
>Priority: Major
> Fix For: 3.6.0
>
> Attachments: Capture_d_écran_de_2020-06-22_12-10-11.png,
> Capture_d_écran_de_2020-06-22_12-11-46.png
>
>
> {{}}
> We are reading system mailboxes to validate each updates.
> What is surprising is the high amount of mailbox reads: we are reading system
> mailboxes to validate each updates.
> The flamegraph correlates positively with the above assertion.
> (see attachments)
> *DoD*
> it should be done only once
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Commented] (JAMES-3277) Don't read outbox mailbox for each message in SetMessagesUpdateProcessor
[
https://issues.apache.org/jira/browse/JAMES-3277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17212137#comment-17212137
]
Benoit Tellier commented on JAMES-3277:
---
Some work remains on this topic...
# Why
{code:java}
ASYNC
Transaction type:
Web
Transaction name:
/jmap
Start:
2020-10-12 9:01:04.358 am (+07:00)
Duration:
4,301.2 milliseconds
Breakdown (Main Thread):total (ms) count
http request
0.431
Breakdown (Auxiliary Threads): total (ms) count
auxiliary thread
6,096.4 12,844
jmapMethod
4,266.4 1
cassandra query
101.6 1,538
Breakdown (Async Timers): total (ms) count
cassandra query
4,470.6 1,538
JVM Thread Stats (Main Thread)
CPU time: 0.41 milliseconds
Blocked time: 0.0 milliseconds
Waited time: 0.0 milliseconds
Allocated memory: 18.5 KB
JVM Thread Stats (Auxiliary Threads)
CPU time: 1,099.3 milliseconds
Blocked time: 0.0 milliseconds
Waited time: 4,769.0 milliseconds
Allocated memory: 28.8 MB
{code}
A query updating 56 messages took 4.3 seconds to complete (76 ms per message!)
raising a total of 1539 Cassandra queries (27 per messages).
This is a recurring pattern of our Cassandra slow logs.
# Details
See attached screenshot
# How
A quick analysis allowed to identify that:
- JAMES-3277 was partially done: an outbox role read remained in
SetMessageUpdateProcessor
- CassandraMessageIdMapper::setFlags was needlessly reading messages twice
- messagev2 table was queried while it is uneeded
- Reading messages individually in SetMessageUpdateProcessor results in right
resolution and existence to be performed twice
- Reading messages collectively enable some sort of parallelism
- StoreMessageIdManager::setFlags was needlessly reading mailboxes twice
# Definition of done
Reduce Cassandra query count upon SetMessages flags update (33%) thus fastening
flags update
# Not in the DOD
Grouping updates by mailboxes in order to allocate the messageId only once and
perform the updates in parallel.
Though significant improvement can be achieved that way!
# Expected outcome
This work enables saving 10 requests per message and according to the execution
timing above should save around 1480ms of Cassandra query time.
> Don't read outbox mailbox for each message in SetMessagesUpdateProcessor
>
>
> Key: JAMES-3277
> URL: https://issues.apache.org/jira/browse/JAMES-3277
> Project: James Server
> Issue Type: Bug
> Components: JMAP
>Affects Versions: 3.5.0
>Reporter: Rémi Kowalski
>Assignee: Antoine Duprat
>Priority: Major
> Fix For: 3.6.0
>
> Attachments: Capture_d_écran_de_2020-06-22_12-10-11.png,
> Capture_d_écran_de_2020-06-22_12-11-46.png
>
>
> {{}}
> We are reading system mailboxes to validate each updates.
> What is surprising is the high amount of mailbox reads: we are reading system
> mailboxes to validate each updates.
> The flamegraph correlates positively with the above assertion.
> (see attachments)
> *DoD*
> it should be done only once
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Commented] (JAMES-3277) Don't read outbox mailbox for each message in SetMessagesUpdateProcessor
[
https://issues.apache.org/jira/browse/JAMES-3277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148531#comment-17148531
]
René Cordier commented on JAMES-3277:
-
[https://github.com/linagora/james-project/pull/3481] has been merged
> Don't read outbox mailbox for each message in SetMessagesUpdateProcessor
>
>
> Key: JAMES-3277
> URL: https://issues.apache.org/jira/browse/JAMES-3277
> Project: James Server
> Issue Type: Bug
> Components: JMAP
>Affects Versions: 3.5.0
>Reporter: Rémi Kowalski
>Assignee: Antoine Duprat
>Priority: Major
> Attachments: Capture_d_écran_de_2020-06-22_12-10-11.png,
> Capture_d_écran_de_2020-06-22_12-11-46.png
>
>
> {{}}
> We are reading system mailboxes to validate each updates.
> What is surprising is the high amount of mailbox reads: we are reading system
> mailboxes to validate each updates.
> The flamegraph correlates positively with the above assertion.
> (see attachments)
> *DoD*
> it should be done only once
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Commented] (JAMES-3277) Don't read outbox mailbox for each message in SetMessagesUpdateProcessor
[
https://issues.apache.org/jira/browse/JAMES-3277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148408#comment-17148408
]
René Cordier commented on JAMES-3277:
-
[https://github.com/linagora/james-project/pull/3478] has been merged
> Don't read outbox mailbox for each message in SetMessagesUpdateProcessor
>
>
> Key: JAMES-3277
> URL: https://issues.apache.org/jira/browse/JAMES-3277
> Project: James Server
> Issue Type: Bug
> Components: JMAP
>Affects Versions: 3.5.0
>Reporter: Rémi Kowalski
>Assignee: Antoine Duprat
>Priority: Major
> Attachments: Capture_d_écran_de_2020-06-22_12-10-11.png,
> Capture_d_écran_de_2020-06-22_12-11-46.png
>
>
> {{}}
> We are reading system mailboxes to validate each updates.
> What is surprising is the high amount of mailbox reads: we are reading system
> mailboxes to validate each updates.
> The flamegraph correlates positively with the above assertion.
> (see attachments)
> *DoD*
> it should be done only once
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Commented] (JAMES-3277) Don't read outbox mailbox for each message in SetMessagesUpdateProcessor
[
https://issues.apache.org/jira/browse/JAMES-3277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146078#comment-17146078
]
Benoit Tellier commented on JAMES-3277:
---
https://github.com/linagora/james-project/pull/3481 reduces the amount of
mailbox reads performed by StoreMessageManager::setInMailboxes
https://github.com/linagora/james-project/pull/3478 avoids reading the outbox
mailbox for each message
> Don't read outbox mailbox for each message in SetMessagesUpdateProcessor
>
>
> Key: JAMES-3277
> URL: https://issues.apache.org/jira/browse/JAMES-3277
> Project: James Server
> Issue Type: Bug
> Components: JMAP
>Affects Versions: 3.5.0
>Reporter: Rémi Kowalski
>Assignee: Antoine Duprat
>Priority: Major
> Attachments: Capture_d_écran_de_2020-06-22_12-10-11.png,
> Capture_d_écran_de_2020-06-22_12-11-46.png
>
>
> {{}}
> We are reading system mailboxes to validate each updates.
> What is surprising is the high amount of mailbox reads: we are reading system
> mailboxes to validate each updates.
> The flamegraph correlates positively with the above assertion.
> (see attachments)
> *DoD*
> it should be done only once
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Commented] (JAMES-3277) Don't read outbox mailbox for each message in SetMessagesUpdateProcessor
[
https://issues.apache.org/jira/browse/JAMES-3277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17144583#comment-17144583
]
Benoit Tellier commented on JAMES-3277:
---
We are in my opinion missing valuable information to understand this work.
> Don't read outbox mailbox for each message in SetMessagesUpdateProcessor
>
>
> Key: JAMES-3277
> URL: https://issues.apache.org/jira/browse/JAMES-3277
> Project: James Server
> Issue Type: Bug
> Components: JMAP
>Affects Versions: 3.5.0
>Reporter: Rémi Kowalski
>Assignee: Antoine Duprat
>Priority: Major
>
> {{}}
> We are reading system mailboxes to validate each updates.
> *DoD*
> it should be done only once
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org
