[
https://issues.apache.org/jira/browse/CASSANDRA-5661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13701480#comment-13701480
]
Jonathan Ellis commented on CASSANDRA-5661:
-------------------------------------------
I talked this over a bit with [~ben.manes] (author of CLHM). Here's his take:
{quote}
[This] sounds less like a multimap cache than a multi-way object pool. To me a
multimap cache would be like any other cache where entries are read frequently,
rarely explicitly invalidated, and evicted by a boundary condition. An object
pool has instances checked in and out, so care needs be taken to make sure the
transfer overhead is cheap. I think you want a more advanced pool with global
boundary conditions and is multi-way, so more complex than a traditional
database connection pool. For that, actually, a few years ago I advised the
author of BoneCP to use a LinkedTransferQueue to leverage elimination to avoid
contention which provided the performance improvements to make his library the
fastest available.
{quote}
Ben put together an implementation at
https://github.com/ben-manes/multiway-pool:
{quote}
* A concurrent object pool that supports pooling multiple resources that are
associated with a
* single key. A resource is borrowed from the pool, used exclusively, and
released back for reuse
* by another caller. This implementation can optionally be bounded by a maximum
size, time-to-live,
* or time-to-idle policies.
* <p>
* A traditional object pool is homogeneous; all of the resources are identical
in the data and
* capabilities offered. For example a database connection pool to a shared
database instance. A
* multiway object pool is heterogeneous; resources may differ in the data and
capabilities offered.
* For example a flat file database may pool random access readers to the
database table files. The
* relationship of a single-way to a multi-way object pool is similar to that of
a map to a
* multimap.
* <p>
* When this pool is bounded any resource is eligible for eviction regardless of
the key that it is
* associated with. A size based bound will evict resources by a best-effort LRU
policy and a time
* based policy will evict by either a time-to-idle and/or time-to-live policy.
The resource's life
* cycle can be instrumented, such as when cleaning up after eviction, by using
the appropriate
* {@link ResourceLifecycle} method.
{quote}
> Discard pooled readers for cold data
> ------------------------------------
>
> Key: CASSANDRA-5661
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5661
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.2.1
> Reporter: Jonathan Ellis
> Assignee: Pavel Yaskevich
> Fix For: 1.2.7
>
> Attachments: CASSANDRA-5661.patch, DominatorTree.png, Histogram.png
>
>
> Reader pooling was introduced in CASSANDRA-4942 but pooled
> RandomAccessReaders are never cleaned up until the SSTableReader is closed.
> So memory use is "the worst case simultaneous RAR we had open for this file,
> forever."
> We should introduce a global limit on how much memory to use for RAR, and
> evict old ones.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
