David Capwell commented on CASSANDRA-15388:

 is packaged protected, would be great to use that to see if the agent is 

To make make things clear, 
 it would be good if you did

diff --git 
index 6394b92e6d..53d11c9031 100644
@@ -74,7 +74,7 @@ public class CompactionAllocationTest
     private static final Logger logger = 
     private static final ThreadMXBean threadMX = (ThreadMXBean) 
-    private static final boolean AGENT_MEASUREMENT = true;
+    private static final boolean AGENT_MEASUREMENT = isAgentLoaded();
     private static final boolean PROFILING_READS = false;
     private static final boolean PROFILING_COMPACTION = false;
@@ -153,6 +153,14 @@ public class CompactionAllocationTest
+    private static boolean isAgentLoaded()
+    {
+        AgentMeasurement measurement = new AgentMeasurement();
+        measurement.start();
+        measurement.stop();
+        return measurement.objectsAllocated != 0 || measurement.bytesAllocated 
!= 0;
+    }
     public static void setupClass() throws Throwable


This way we only use the agent if loaded.  Right now you don't modify the IDE 
files, so if you run the test in any IDE it will always log that you allocated 
0 bytes; with the patch I posted it uses the thread mbean.

 Make this overridable; a system properly would be fine

 remove the branch or get rid of "|| true"

 Why all the sleeps?  If I remove them I don't really see a difference

INFO  [main] 2020-02-14 16:24:20,357 CompactionAllocationTest.java:433 - *** 
tinyNonOverlapping3 reads summary
INFO  [main] 2020-02-14 16:24:20,357 CompactionAllocationTest.java:434 - 
11592072 bytes, 4293 /read, 116490000 cpu
INFO  [main] 2020-02-14 16:24:20,357 CompactionAllocationTest.java:435 - *** 
tinyNonOverlapping3 compaction summary
INFO  [main] 2020-02-14 16:24:20,357 CompactionAllocationTest.java:436 - 
10893688 bytes, 4034 /partition, 4034 /row, 93812000 cpu


INFO  [main] 2020-02-14 16:25:17,491 CompactionAllocationTest.java:433 - *** 
tinyNonOverlapping3 reads summary
INFO  [main] 2020-02-14 16:25:17,491 CompactionAllocationTest.java:434 - 
11589808 bytes, 4292 /read, 109551000 cpu
INFO  [main] 2020-02-14 16:25:17,491 CompactionAllocationTest.java:435 - *** 
tinyNonOverlapping3 compaction summary
INFO  [main] 2020-02-14 16:25:17,491 CompactionAllocationTest.java:436 - 
10893848 bytes, 4034 /partition, 4034 /row, 74584000 cpu

The only real change I see is in terms of CPU, which makes sense since you just 
track total time (though implemented from mbean).

The only hint I have as to why is the comment "// maybe log entries will stop 
disappearing?"  which makes me think logger?  


Looks like you are trying to create a report, but targeting humans?  Can you 
move the strings out of the logger and into something like StringBuilder?  Once 
you complete you can add to the logger for now (though more value if you 
generate a test report, but fine if this is a different JIRA)

Last comment, it would be nice if you asserted the bounds of memory allocated 
rather than log.

> Add compaction allocation measurement test to support compaction gc 
> optimization. 
> ----------------------------------------------------------------------------------
>                 Key: CASSANDRA-15388
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-15388
>             Project: Cassandra
>          Issue Type: Sub-task
>          Components: Local/Compaction
>            Reporter: Blake Eggleston
>            Assignee: Blake Eggleston
>            Priority: Normal
>             Fix For: 4.0
> This adds a test that is able to quickly and accurately measure the effect of 
> potential gc optimizations against a wide range of (synthetic) compaction 
> workloads. This test accurately measures allocation rates from 16 workloads 
> in less that 2 minutes.
> This test uses google’s {{java-allocation-instrumenter}} agent to measure the 
> workloads. Measurements using this agent are very accurate and pretty 
> repeatable from run to run, with most variance being negligible (1-2 bytes 
> per partition), although workloads with larger but fewer partitions vary a 
> bit more (still less that 0.03%).
> The thinking behind this patch is that with compaction, we’re generally 
> interested in the memory allocated per partition, since garbage scales more 
> or less linearly with the number of partitions compacted. So measuring 
> allocation from a small number of partitions that otherwise represent real 
> world use cases is a good enough approximation.
> In addition to helping with compaction optimizations, this test could be used 
> as a template for future optimization work. This pattern could also be used 
> to set allocation limits on workloads/operations and fail CI if the 
> allocation behavior changes past some threshold. 

This message was sent by Atlassian Jira

To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

Reply via email to