David Capwell updated CASSANDRA-15388:
    Status: Changes Suggested  (was: Review In Progress)

> Add compaction allocation measurement test to support compaction gc 
> optimization. 
> ----------------------------------------------------------------------------------
>                 Key: CASSANDRA-15388
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-15388
>             Project: Cassandra
>          Issue Type: Sub-task
>          Components: Local/Compaction
>            Reporter: Blake Eggleston
>            Assignee: Blake Eggleston
>            Priority: Normal
>             Fix For: 4.0
> This adds a test that is able to quickly and accurately measure the effect of 
> potential gc optimizations against a wide range of (synthetic) compaction 
> workloads. This test accurately measures allocation rates from 16 workloads 
> in less that 2 minutes.
> This test uses google’s {{java-allocation-instrumenter}} agent to measure the 
> workloads. Measurements using this agent are very accurate and pretty 
> repeatable from run to run, with most variance being negligible (1-2 bytes 
> per partition), although workloads with larger but fewer partitions vary a 
> bit more (still less that 0.03%).
> The thinking behind this patch is that with compaction, we’re generally 
> interested in the memory allocated per partition, since garbage scales more 
> or less linearly with the number of partitions compacted. So measuring 
> allocation from a small number of partitions that otherwise represent real 
> world use cases is a good enough approximation.
> In addition to helping with compaction optimizations, this test could be used 
> as a template for future optimization work. This pattern could also be used 
> to set allocation limits on workloads/operations and fail CI if the 
> allocation behavior changes past some threshold. 

This message was sent by Atlassian Jira

To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

Reply via email to