[
https://issues.apache.org/jira/browse/CASSANDRA-21287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18079871#comment-18079871
]
Dmitry Konstantinov commented on CASSANDRA-21287:
-------------------------------------------------
Test details:
* Env
** 1-node Cassandra cluster
*** m8i.4xlarge, CPU: Intel Xeon 16 vCPU, RAM: 64GiB
*** disk: gp3, IOPS limit: 3000, throughput: 200 (more important for write
tests, in read tests I rely on a warm page cache)
*
** Load generator node
*** cassandra-stress (yep, I know it is time to adopt easy stress finally :) )
*** c5.9xlarge, CPU: Intel Xeon 36 vCPU, RAM: 72 GiB
* Preliminary test data generation - insert data and wait for compactions to
complete
{code:java}
./tools/bin/cassandra-stress "user profile=./insert_profile_seq.yaml no-warmup
ops(insert=1,partition-select=0) n=15m" -rate threads=300 -node $CASS_IP -mode
native cql3 maxPending=256 connectionsPerHost=16
{code}
* Test load - select partition (~10 rows, 5 value columns), warm page cache
(to focus on CPU/memory optimizations, not IO)
{code:java}
./tools/bin/cassandra-stress "user profile=./insert_profile_seq.yaml no-warmup
ops(insert=0,partition-select=1) n=15m" -rate threads=300 -node $CASS_IP -mode
native cql3 maxPending=256 connectionsPerHost=16
{code}
** config: [^insert_profile_seq.yaml]
* Profiles capturing (async-profiler-4.2.1) - cpu/wall/alloc in heatmap mode
./profile_mem.sh
{code:java}
#!/bin/bash
PROFILE_DURATION_SEC_DEFAULT=120
PROFILE_NAME_DEFAULT=prof
PROFILE_DURATION_SEC=${2:-$PROFILE_DURATION_SEC_DEFAULT}
PROFILE_NAME=${1:-$PROFILE_NAME_DEFAULT}
CASSANDRA_PID=$(ps uax | grep CassandraDaemon | grep -v grep | awk '{print $2}')
echo "run profiling for PID=$CASSANDRA_PID, duration: $PROFILE_DURATION_SEC
sec, profile name: $PROFILE_NAME"
./bin/asprof -d $PROFILE_DURATION_SEC -i 5ms -f ${PROFILE_NAME}.jfr -e
cpu,wall,alloc -F vtable $CASSANDRA_PID
#./bin/asprof -d $PROFILE_DURATION_SEC -i 5ms -f ${PROFILE_NAME}.jfr -e alloc
$CASSANDRA_PID
./bin/jfrconv --cpu -o heatmap $PROFILE_NAME.jfr ${PROFILE_NAME}_cpu.html
./bin/jfrconv --wall -o heatmap $PROFILE_NAME.jfr ${PROFILE_NAME}_wall.html
./bin/jfrconv --alloc -o heatmap $PROFILE_NAME.jfr ${PROFILE_NAME}_alloc.html
{code}
> Reduce memory allocation in Cassandra read path
> -----------------------------------------------
>
> Key: CASSANDRA-21287
> URL: https://issues.apache.org/jira/browse/CASSANDRA-21287
> Project: Apache Cassandra
> Issue Type: Epic
> Components: Consistency/Coordination, Local/Memtable, Local/SSTable
> Reporter: Dmitry Konstantinov
> Assignee: Dmitry Konstantinov
> Priority: Normal
> Fix For: 6.x, 7.x
>
> Attachments: insert_profile_seq.yaml
>
>
> The epic accumulates improvements about reducing of memory allocations in
> Cassandra read path.
> The drivers for this epic are:
> * to reduce GC pressure -> reduce GC overhead and GC pauses
> frequency/duration. In case of low-latency GC reducing of memory allocation
> also helps to move forward the threshold when the system throughput is
> limited by the memory allocation throughput a GC is able to process without
> slowing down an app (allocation stalls).
> * memory allocations also help to highlight the places to optimize from CPU
> usage point of view by just doing less work
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
