Benoit Tellier created JAMES-3506:
-------------------------------------
Summary: SMTP stach is slow and generate high GC when under high
traffic
Key: JAMES-3506
URL: https://issues.apache.org/jira/browse/JAMES-3506
Project: James Server
Issue Type: Improvement
Components: Blob, mailbox, SMTPServer
Reporter: Benoit Tellier
Fix For: 3.6.0
We had been documenting that SMTP stack is slow and generate a lot of GC
(https://github.com/apache/james-project/pull/309#issuecomment-786358253)
Under the hood, in memory structures are being used by the distributed server
(MailStore, for both enqueuing and dequeuing emails), copies are enforced by
inefficient APIs (eg InputStream) that prevents replaying the content (while we
can!) and forces defensive copies.
Out of this diagnosis, I started experimenting in
https://github.com/apache/james-project/pull/309 trying to apply the following
principles on the SMTP write path:
- Avoid holding in memory data structures
- Avoid copies by allowing stream generation
My work show that I achieved a x3 SMTP throughput improvement. All dbs,
including a zenko S3 server, being hosted on the same server of my test infra,
I expect the gains to be even higher for real deployments.
This work on efficiency should largely outweight the performance impacts of
JAMES-3477.
I would wish this work makes it to the future release.
On the upcoming topics of attention in my head that might see related works is
the APPEND command buggy inMemorySize limits (exceeding the size limit causes
the APPEND to crash), thus as a temporary remediation we did enforce a higher
memory limit, hence defeating the above mentioned principles. I would prefer
seeing there a FileBackedOutputStream with a replayable byte source, achieving
similar enhencements for the APPEND command.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
