[
https://issues.apache.org/jira/browse/CASSANDRA-6689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13919225#comment-13919225
]
Benedict commented on CASSANDRA-6689:
-------------------------------------
bq. What I want is to see more control over how we deal with cleaner threads,
as the main point to be able to run with smaller heaps, and thread stacks could
consume ~300K we are risking to OOM in certain situations because we don't
control whole thread life time, that would also help for people in environments
guided by ulimit or similar.
Just for clarity, it's the _collector_ threads, not the _cleaner_ threads
you're worried about, right? Because there is only ever one cleaner thread.
Like I say, this is mostly trying to help with an anti-pattern, so I'm not sure
it's worth addressing too aggressively in this ticket. In real terms you'll
struggle to spin up very many threads at all, but let's put an arbitrary limit
of 100 threads on the pool for now. I don't think this is worth offering
configuration options to the user here.
bq. Please do think about it, we need to try to reduce the complexity of this
to minimize bus factor
As I said, I (still) can't see a good way to use this information to simplify
matters usefully. If we flattened Pool, AllocatorGroup and Allocator all into
the one object, and we had a per-CF memory limit, we would be able to simplify
all of the SubPool and SubAllocator memory management considerably, but then we
have the issue that there's no global management of memory resources, nor
shared re-use. I don't think this would simplify the actual recycling of memory
regions that is introduced in this ticket at all though, but it's possible I'm
missing something.
Suggestions always welcome :-)
bq. Exactly, maybe it's time to re-evaluate trade-off between copying data to
per-allocated regions with limited ttl vs. tracking data all around the place.
This is the opposite suggestion, right? :-) It is definitely something worth
exploring in future.
bq. Have you had a chance to do more on vs. off heap performance testing?
I'm not sure Marcus was performance testing. Personally I'm most interested in
correctness right now: performance should follow (in particular, slow flushing
is certainly a "bug" which can be fixed). Certainly for overwrite workloads
I've demonstrated the reduced flushing on my box, but further testing of the
impact is likely to be a long drawn out process, and probably beyond scope of
this ticket IMO.
Performance testing scope might include:
- Establish how much heap limit reduction this permits
- Establish a severe GC pause inducing workload to see if this mitigates
- Establish if G1GC is made viable with this, and if not, what remaining
impacts there are to adoption that might be lifted
- Establish how much improvement to overwrite workloads (or mixed workloads,
where reduced overwrite workload memory consumption => reduced non-overwrite
workload flushing)
- Establish new reasonable upper memory limit for heap and memtable (both
independently), for different workloads. e.g. large data payload workloads may
now practicably support much larger memtables and dramatically reduced flushing
- Establish impact on upper *data* limit for different workloads, with same
memory limit as olde, and the impact on size-of-flush and concomitant reduction
in write amplification
> Partially Off Heap Memtables
> ----------------------------
>
> Key: CASSANDRA-6689
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6689
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Benedict
> Assignee: Benedict
> Fix For: 2.1 beta2
>
> Attachments: CASSANDRA-6689-small-changes.patch
>
>
> Move the contents of ByteBuffers off-heap for records written to a memtable.
> (See comments for details)
--
This message was sent by Atlassian JIRA
(v6.2#6252)
