[
https://issues.apache.org/jira/browse/CASSANDRA-15512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17041353#comment-17041353
]
Blake Eggleston commented on CASSANDRA-15512:
---------------------------------------------
This is really cool. I’ve spent some time working on the parameter space that
these benchmarks work on, the run time, and playing around with ways to
organize the large output into something that can be summarized and explored a
bit more easily.
My branch is here: https://github.com/bdeggleston/cassandra/tree/15512-trunk,
let me know what you think. Changes described below.
*Workload Normalization*
Instead of individual parameter ranges for partition, row, and column counts,
there’s now a single parameter (concentration) that sweeps from 0.0 to 1.0, and
the partition/row/column counts are derived from that. As this parameter
changes, the values of partitions, rows, and columns change, but the result of
(partitions * rows * columns) stays about the same, so you’re (in theory)
comparing the same volume of work, just done by different parts of the read
path.
This has been really useful for getting a high level picture of the performance
impact of a set of changes over a variety of workloads. That said, it does this
at the expense of seeing how more extreme workloads perform, or comparing the
throughput of workloads moving different amounts of data (measuring column
index overhead, for example), but that seems like something we should add
later, instead of trying to also do it here.
*Result Visualization*
I added a script that generates a heat map of the relative throughput values
for each parameter permutation, as well as the difference between 2 runs for a
quick before and after comparison. By graphing the concentration on the x axis,
and column_type -> value size -> overlap on the y axis, you can quickly get an
idea of how things change, both per run and between them.
*Other Stuff*
I added a micro bench test script that sets up the JVM in about the same way as
the server config. There may be a better way to do this, but this has worked
fine for me.
There's also a handful of review fix style changes:
* I fixed a few bugs I found
* Switched assert in CompactionController ctor to use Preconditions.
* added @VisibleForTesting annotation to PartitionUpdate#unsafeConstruct
* cleaned up some imports
*Misc WIP*
There are also a handful of tools that aren't really at ready-to-commit levels
of polish, but that have been useful. Some of these may be worth committing in
their incomplete state so that they will continue to improve as part of our
performance tooling. I might also be able to put them in a repo where they
could be worked on until they're "good enough"
* A profiler that includes jvm methods (ie: gc), and kernel activity in the
results. It uses the profile tool from bcc-tools, modified to extract symbols
for jit compiled methods from the jvm before dumping its data. No idea if it
would be possible/practical to eventually be able to map interpreter stack
frames to their java class/method. Anyway, it's still pretty rough, and needs a
bit of setup, but helps put things in perspective when working on things that
are focused on reducing gc time. It's basically like perf record, except the
sampled data is aggregated in the kernel, and data is only sent to user space
at the end of the profiling session.
* A perf profiler that profiles cpu performance counters, which I used to
profile branch misses. I'd like to rewrite this as a bpf script, possibly as an
extension to the profiler I was just talking about, or modify stackcount from
bcc-tools.
* Flame graph generator. I'd originally written this so I could embed
comparative flame graphs in the heatmap for each run. Unfortunately the
generator is too slow, and it's output is too large for this to be currently
practical. The comparative part also wasn't as useful as I'd hoped.
> Add compaction-related JMH tests
> --------------------------------
>
> Key: CASSANDRA-15512
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15512
> Project: Cassandra
> Issue Type: Sub-task
> Components: Local/Compaction
> Reporter: Benedict Elliott Smith
> Assignee: Benedict Elliott Smith
> Priority: Normal
>
> Following up CASSANDRA-15388, we should introduce some isolated JMH tests
> covering at least the compaction merge path and partition deserialization,
> covering a wide range of partition shapes and characteristics.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
