[
https://issues.apache.org/jira/browse/SOLR-10205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15897721#comment-15897721
]
Yonik Seeley edited comment on SOLR-10205 at 3/6/17 5:48 PM:
-------------------------------------------------------------
Here's the results (attached) of testing with different numbers of reserved
blocks (up to 4) and different number of calls to cleanUp when the map size
exceeds the number of blocks - reserved. Tests were done on systems with 16
and 32 logical (hyper-threaded) cores.
The speedups compared to trunk range from 11% to 68% for these artificial
random tests.
Based on the results, I think the right balance is going with reserved blocks =
4 and a single call to cleanUp in the outer loop of
BlockCache.findEmptyLocation()
was (Author: [email protected]):
Here's the results of testing with different numbers of reserved blocks (up to
4) and different number of calls to cleanUp when the map size exceeds the
number of blocks - reserved.
The speedups compared to trunk range from 11% to 68% for these artificial
random tests.
Based on the results, I think the right balance is going with reserved blocks =
4 and a single call to cleanUp in the outer loop of
BlockCache.findEmptyLocation()
> Evaluate and reduce BlockCache store failures
> ---------------------------------------------
>
> Key: SOLR-10205
> URL: https://issues.apache.org/jira/browse/SOLR-10205
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Reporter: Yonik Seeley
> Assignee: Yonik Seeley
> Attachments: cache_performance_test.txt, SOLR-10205.patch,
> SOLR-10205.patch
>
>
> The BlockCache is written such that requests to cache a block
> (BlockCache.store call) can fail, making caching less effective. We should
> evaluate the impact of this storage failure and potentially reduce the number
> of storage failures.
> The implementation reserves a single block of memory. In store, a block of
> memory is allocated, and then a pointer is inserted into the underling map.
> A block is only freed when the underlying map evicts the map entry.
> This means that when two store() operations are called concurrently (even
> under low load), one can fail. This is made worse by the fact that
> concurrent maps typically tend to amortize the cost of eviction over many
> keys (i.e. the actual size of the map can grow beyond the configured maximum
> number of entries... both the older ConcurrentLinkedHashMap and newer
> Caffeine do this). When this is the case, store() won't be able to find a
> free block of memory, even if there aren't any other concurrently operating
> stores.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
