[
https://issues.apache.org/jira/browse/SOLR-10141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15873334#comment-15873334
]
Ben Manes commented on SOLR-10141:
----------------------------------
If you wish to ensure a very strict bounding by throttling writers, that would
do the job. I'm not sure if its needed except in your tests, as in practice the
assumption is its cleaned up in a timely enough manner.
The cache uses a bounded write buffer to provide some slack, minimize the
response latencies for writers, and defers the cleanup to the executor
(scheduled as immediate). This allows the cache to temporarily exceed the high
water mark, but catch up quickly. In general a high write rate on a cache is
actually 2-3 inserts/sec, there's memory headroom for GC, and the server isn't
cpu bounded. If instead we ensured a strict bound then we'd need a global lock
to throttle writers on which limits concurrency. So its a trade-off that works
for most usages.
CLHM uses the same design, so I wonder if only your tests are affected but it
is okay in practice. CLHM uses an unbounded write buffer, whereas in Caffeine
its bounded to provide some back pressure if full. Being full is very rare, so
this is mostly to replace linked lists with a growable ring buffer. The slack
is probably excessive as I didn't have a good sizing parameter (max ~= 128 x
ncpu). The cleanUp() call forces the caller to block and do the maintenance
itself, rather than relying on the async processing (which may be in-flight or
triggered on a subsequent operation). You can get a sense of this write-ahead
log design from this [slide
deck|https://docs.google.com/presentation/d/1NlDxyXsUG1qlVHMl4vsUUBQfAJ2c2NsFPNPr2qymIBs].
I'm not sure what, or if, I can do anything regarding your size concern. But
I'll wait for releasing 2.4 until you're satisfied that we've resolved all the
issues.
> Caffeine cache causes BlockCache corruption
> --------------------------------------------
>
> Key: SOLR-10141
> URL: https://issues.apache.org/jira/browse/SOLR-10141
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Reporter: Yonik Seeley
> Attachments: SOLR-10141.patch, Solr10141Test.java
>
>
> After fixing the race conditions in the BlockCache itself (SOLR-10121), the
> concurrency test passes with the previous implementation using
> ConcurrentLinkedHashMap and fail with Caffeine.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
