[
https://issues.apache.org/jira/browse/CASSANDRA-21656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jon Haddad reassigned CASSANDRA-21656:
--------------------------------------
Assignee: Jon Haddad
> large mutations over-allocate on the commit log path
> ----------------------------------------------------
>
> Key: CASSANDRA-21656
> URL: https://issues.apache.org/jira/browse/CASSANDRA-21656
> Project: Apache Cassandra
> Issue Type: Bug
> Reporter: Jon Haddad
> Assignee: Jon Haddad
> Priority: Normal
>
> Writing a large mutation to the commit log allocates far more memory than the
> mutation itself, and it does so on every write. The allocation is native, not
> heap, and it grows with mutation size. Mutations under about 1 MiB are
> unaffected.
> Measured on a 4 MiB mutation, 2400 writes, single thread, memory-mapped
> segments, {{cassandra-6.0}} at {{{}38a7a783{}}}, async-profiler {{{}-e
> nativemem{}}}:
> ||Measure||Value||
> |native memory per write|10.42 MB|
> |heap bytes per write|958.7|
> |writes/sec, median of 5|850.5|
> The commit log buffers the serialized mutation before copying it into the
> segment, and the buffer is rebuilt from scratch on each large write. The
> serialized size is already computed earlier in the same call and then
> discarded, so it may be possible to size the segment slot from it and
> serialize once, directly. Whether that size is exact enough to be trusted for
> the slot is the open question.
> I have a branch (wip) that removes the intermediate buffer. The serialized
> size is already computed earlier in the same call, so the segment slot is
> sized from it and the mutation is serialized straight into it.
> Here are some stats:
>
> ||Measure||Before||After||Change||
> |Native memory per write, 4 MiB mutation|10.42 MB|19.0 KB|-99.8%|
> |Native memory total, 2400 writes|23.30 GiB|43.6 MiB|-99.8%|
> |Heap bytes per write, 4 MiB mutation|958.7|547.3|-42.9%|
> |Heap bytes per write, 200 B mutation|161.7|161.6|not resolved|
> The branch is a little early, but it's my hope that this will help reduce the
> pain people experience when writing large objects.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
