[
https://issues.apache.org/jira/browse/CASSANDRA-15229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17074656#comment-17074656
]
Benedict Elliott Smith commented on CASSANDRA-15229:
----------------------------------------------------
bq. a bump-the-pointer slab approach for the transient pool, not to dissimilar
from the current implementation. We then exploit our thread per core
architecture: core threads get a dedicated slab each, other threads share a
global slab.
The current implementation isn't really a bump the pointer allocator? It's
bitmap based, though with a very tiny bitmap. Could you elaborate on how these
work, as my intuition is that anything designed for a thread-per-core
architecture probably won't translate so well to the present state of the
world. Though, either way, I suppose this is probably orthogonal to this
ticket as we only need to address the {{ChunkCache}} part.
bq. We also optimized the chunk cache to store memory addresses rather than
byte buffers, which significantly reduced heap usage. The byte buffers are
materialized on the fly.
This would be a huge improvement, and a welcome backport if it is easy - though
it might (I would guess) depend on {{Unsafe}}, which may be going away soon.
It's orthogonal to this ticket, though, I think.
bq. We changed the chunk cache to always store buffers of the same size.
bq. We have global lists of these slabs, sorted by buffer size where each size
is a power-of-two.
How do these two statements reconcile?
Is it your opinion that your entire {{ChunkCache}} implementation can be
dropped wholesale into 4.0? I would assume it is still primarily
multi-threaded. If so, it might be preferable to trying to fix the existing
{{ChunkCache}}
> BufferPool Regression
> ---------------------
>
> Key: CASSANDRA-15229
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15229
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Caching
> Reporter: Benedict Elliott Smith
> Assignee: ZhaoYang
> Priority: Normal
> Fix For: 4.0, 4.0-beta
>
>
> The BufferPool was never intended to be used for a {{ChunkCache}}, and we
> need to either change our behaviour to handle uncorrelated lifetimes or use
> something else. This is particularly important with the default chunk size
> for compressed sstables being reduced. If we address the problem, we should
> also utilise the BufferPool for native transport connections like we do for
> internode messaging, and reduce the number of pooling solutions we employ.
> Probably the best thing to do is to improve BufferPool’s behaviour when used
> for things with uncorrelated lifetimes, which essentially boils down to
> tracking those chunks that have not been freed and re-circulating them when
> we run out of completely free blocks. We should probably also permit
> instantiating separate {{BufferPool}}, so that we can insulate internode
> messaging from the {{ChunkCache}}, or at least have separate memory bounds
> for each, and only share fully-freed chunks.
> With these improvements we can also safely increase the {{BufferPool}} chunk
> size to 128KiB or 256KiB, to guarantee we can fit compressed pages and reduce
> the amount of global coordination and per-allocation overhead. We don’t need
> 1KiB granularity for allocations, nor 16 byte granularity for tiny
> allocations.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
