chibenwa opened a new pull request #434:
URL: https://github.com/apache/james-project/pull/434
Using metrics to capture p99 lead to over-snapshoting, and incurs a 33%
throughtput penalty on JMAP draft.
Other SLOW logging implementations do rely on manual time
measurements (eg Datastax Cassandra driver).
As such, due to the low benefits and high costs I propose
to deprecate the logP99 API and migrate away from it.
Tools like Glowroot are able to capture slow traces at a
lighter cost and should rather be used.
## Performance
I did run 2 benchmarks to evaluate the changes in terms of performance:
- a. Run 1000 users doing a representative JMAP polling request with a
pause of 2-4 seconds in between. This represents the regular usage.
-b. Run 2000 users doing a representative JMAP polling request with a
pause of 2-4 seconds in between. This adapts to system maximum throughtput (as
high latencies would add up to the delays and acts as a feedback loop).
For each of these scenari we include:
- Gatling overview
- Glowroot average, that includes time spent executing Cassanra queries
- Glowroot percentiles
- Glowroot averages (1 min)
- And finally Glowroot Cassandra queries breakdown
Testing infrastructure:
- 3 James, 3 CPU, 6GB memory
- 3 Cassandra (SSD, 8 CPU, 32GB ram) - OVH B2-30)
- OVH S3 object store
The instances are pre-provisionned with representative email corpus. Each
account have a representative mailbox distribution 50 -> 1000 mailboxes (hence
Mailbox/get dispersion)
### Before
#### a. LOG p99 and Regular scenario
#### b. LOG p99 and maximum throughtput
### After
#### a. LOG p99 and Regular scenario
#### b. LOG p99 and maximum throughtput
### Conclusion
- Dramatic improvments of all KPIs at all levels of load. We can reach
workloads with +50% higher
- Surprisingly enough, Cassandra queries executed faster. I think because
of CPU shortage Cassandra query execution lead to delays.
Side note: I tried many variations of the metrics, and oddly enough the
final one behave even better than a "logging" metric implementation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
