[
https://issues.apache.org/jira/browse/CASSANDRA-15393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16963950#comment-16963950
]
Benedict Elliott Smith edited comment on CASSANDRA-15393 at 10/31/19 12:34 PM:
-------------------------------------------------------------------------------
Have we considered, for compaction, materialising {{NativeCell}} from disk
instead? These have only 16 or 24 bytes of heap overhead, depending on the JVM
settings, and avoid materialising any of the payload. We would need to
parameterise our {{Serializer}}, and for compaction associate it with a
{{NativeAllocator}} for producing {{NativeCell}}, and reclaim this after each
partition (or each row of a partition). Even the {{NativeCell}} themselves
could be reused in this scheme, giving zero {{Cell}} related garbage.
Not presupposing this would be trivial, of course, but it is probably not too
difficult - it has the advantage of avoiding any of the above worry of
ramifications outside of the scope of compaction, and has a big payoff.
was (Author: benedict):
Have we considered, for compaction, materialising {{NativeCell}} from disk
instead? These have only 16 or 24 bytes of heap overhead, depending on the JVM
settings, and avoid materialising any of the payload. We would need to
parameterise our {{Serializer}}, and for compaction associate it with a
{{NativeAllocator}} for producing {{NativeCell}}, and reclaim this after each
partition (or each row of a partition). Even the {{NativeCell}} themselves
could be reused in this scheme, giving zero {{Cell}} related garbage.
> Add byte array backed cells
> ---------------------------
>
> Key: CASSANDRA-15393
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15393
> Project: Cassandra
> Issue Type: Sub-task
> Components: Local/Compaction
> Reporter: Blake Eggleston
> Assignee: Blake Eggleston
> Priority: Normal
> Fix For: 4.0
>
>
> We currently materialize all values as on heap byte buffers. Byte buffers
> have a fairly high overhead given how frequently they’re used, and on the
> compaction and local read path we don’t do anything that needs them. Use of
> byte buffer methods only happens on the coordinator. Using cells that are
> backed by byte arrays instead in these situations reduces compaction and read
> garbage up to 22% in many cases.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
