Blake Eggleston created CASSANDRA-15388:
-------------------------------------------
Summary: Add compaction allocation measurement test to support
compaction gc optimization.
Key: CASSANDRA-15388
URL: https://issues.apache.org/jira/browse/CASSANDRA-15388
Project: Cassandra
Issue Type: Sub-task
Components: Local/Compaction
Reporter: Blake Eggleston
Assignee: Blake Eggleston
This adds a test that is able to quickly and accurately measure the effect of
potential gc optimizations against a wide range of (synthetic) compaction
workloads. This test accurately measures allocation rates from 16 workloads in
less that 2 minutes.
This test uses google’s {{java-allocation-instrumenter}} agent to measure the
workloads. Measurements using this agent are very accurate and pretty
repeatable from run to run, with most variance being negligible (1-2 bytes per
partition), although workloads with larger but fewer partitions vary a bit more
(still less that 0.03%).
The thinking behind this patch is that with compaction, we’re generally
interested in the memory allocated per partition, since garbage scales more or
less linearly with the number of partitions compacted. So measuring allocation
from a small number of partitions that otherwise represent real world use cases
is a good enough approximation.
In addition to helping with compaction optimizations, this test could be used
as a template for future optimization work. This pattern could also be used to
set allocation limits on workloads/operations and fail CI if the allocation
behavior changes past some threshold.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
