[
https://issues.apache.org/jira/browse/SOLR-6521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14135713#comment-14135713
]
Jessica Cheng Mallet commented on SOLR-6521:
--------------------------------------------
I worry that this will not be performant enough since that makes lookups of
different keys serial, and in a system with lots of active collections, lookups
will be unnecessarily blocked.
In the same vein, I feel that the codebase can benefit from a
LoadingCache-style cache that only synchronizes loading of the same key.
Incidentally we also found that the FilterCache population suffers from the
same problem. We've found that in our load test, when a filter query is heavily
used, it actually adversely impacts performance to cache the fq because
whenever the filter cache is invalidated, all the request threads try to build
the cache entry for that exact same fq and locks the system up. We would get 30
second 99% request time when we cache the fq, and when we added {!cache=false}
to the fq, 99% went back down to the 100ms range.
I understand that adding guava to solrj would probably require a lot of
discussions and voting, etc., but I think the benefit of a LoadingCache-style
cache is high enough that even if we can't include guava, it might make sense
for solr/apache to implement their own.
> CloudSolrServer should synchronize cache cluster state loading
> --------------------------------------------------------------
>
> Key: SOLR-6521
> URL: https://issues.apache.org/jira/browse/SOLR-6521
> Project: Solr
> Issue Type: Improvement
> Components: SolrCloud
> Reporter: Jessica Cheng Mallet
> Assignee: Noble Paul
> Labels: SolrCloud
>
> Under heavy load-testing with the new solrj client that caches the cluster
> state instead of setting a watcher, I started seeing lots of zk connection
> loss on the client-side when refreshing the CloudSolrServer
> collectionStateCache, and this was causing crazy client-side 99.9% latency
> (~15 sec). I swapped the cache out with guava's LoadingCache (which does
> locking to ensure only one thread loads the content under one key while the
> other threads that want the same key wait) and the connection loss went away
> and the 99.9% latency also went down to just about 1 sec.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
