Hi Shalin,
these are stats for caches used:
*documentCache*
class:org.apache.solr.search.LRUCache
description:LRU Cache(maxSize=128, initialSize=128)
stats:
CACHE.searcher.documentCache.cumulative_evictions:234923643
CACHE.searcher.documentCache.cumulative_hitratio:0
CACHE.searcher.documentCache.cumulative_hits:876732
CACHE.searcher.documentCache.cumulative_inserts:234997370
CACHE.searcher.documentCache.cumulative_lookups:235874102
CACHE.searcher.documentCache.evictions:2123759
CACHE.searcher.documentCache.hitratio:0
CACHE.searcher.documentCache.hits:3209
CACHE.searcher.documentCache.inserts:2124008
CACHE.searcher.documentCache.lookups:2127217
CACHE.searcher.documentCache.size:128
CACHE.searcher.documentCache.warmupTime:0
*filterCache*
class:org.apache.solr.search.FastLRUCache
description:Concurrent LRU Cache(maxSize=128, initialSize=128,
minSize=115, acceptableSize=121, cleanupThread=false, autowarmCount=64,
regenerator=org.apache.solr.search.SolrIndexSearcher$2@418597da)
stats:
CACHE.searcher.filterCache.cumulative_evictions:33848
CACHE.searcher.filterCache.cumulative_hitratio:1
CACHE.searcher.filterCache.cumulative_hits:79684607
CACHE.searcher.filterCache.cumulative_inserts:44931
CACHE.searcher.filterCache.cumulative_lookups:79729534
CACHE.searcher.filterCache.evictions:59
CACHE.searcher.filterCache.hitratio:1
CACHE.searcher.filterCache.hits:708519
CACHE.searcher.filterCache.inserts:118
CACHE.searcher.filterCache.lookups:708637
CACHE.searcher.filterCache.size:123
CACHE.searcher.filterCache.warmupTime:52330
*queryResultCache*
class:org.apache.solr.search.LRUCache
description:LRU Cache(maxSize=256, initialSize=256, autowarmCount=64,
regenerator=org.apache.solr.search.SolrIndexSearcher$3@124adf10)
stats:
CACHE.searcher.queryResultCache.cumulative_evictions:38463897
CACHE.searcher.queryResultCache.cumulative_hitratio:0.01
CACHE.searcher.queryResultCache.cumulative_hits:271649
CACHE.searcher.queryResultCache.cumulative_inserts:38638030
CACHE.searcher.queryResultCache.cumulative_lookups:38909216
CACHE.searcher.queryResultCache.evictions:351561
CACHE.searcher.queryResultCache.hitratio:0.01
CACHE.searcher.queryResultCache.hits:1952
CACHE.searcher.queryResultCache.inserts:353004
CACHE.searcher.queryResultCache.lookups:354874
CACHE.searcher.queryResultCache.size:256
CACHE.searcher.queryResultCache.warmupTime:11724
On 29.10.18 09:51, Shalin Shekhar Mangar wrote:
What does your cache statistics look like? What's the hit ratio, size,
evictions etc?
More comments inline:
On Sat, Oct 27, 2018 at 8:23 AM Erick Erickson <erickerick...@gmail.com>
wrote:
Sofiya:
I haven't said so before, but it's a great pleasure to work with
someone who's done a lot of homework before pinging the list. The only
unfortunate bit is that it usually means the simple "Oh, I can fix
that without thinking about it much" doesn't work ;)
2. I'll clarify a bit here. Any TLOG replica can become the leader.
Here's the process for an update:
doc comes in to the leader (may be TLOG)
doc is forwarded to all TLOG replicas, _but it is not indexed there_.
If the leader fails, the other TLOG replicas have enough documents in
_their_ tlogs to "catch up" and one is elected
You're totally right that PULL replicas cannot become leaders
having all TLOG replicas means that the CPU cycles otherwise consumed by
indexing are available for query processing.
The point here is that TLOG replicas don't need to expend CPU cycles
to index documents, freeing up all those cycles for serving queries.
Now, that said you report that QPS rate doesn't particularly seem to
be affected by whether you're indexing or not, so that makes using
TLOG and PULL replicas less likely to solve your problem. I was
thinking about your statement that you index as fast as possible....
6. This is a little surprising. Here's my guess: You're indexing in
large batches and the batch is only really occupying a thread or two
so it's effectively serialized thus not consuming a huge amount of
resources.
The CloudSolrClient parallelizes updates to each shard leader. But in this
case, there is only 1 shard so all updates are serialized. All indexing
activity is therefore being performed by a single CPU at a time.
So unless G1 really solves a lot of problems, more replicas are
indicated. On machines with large amounts of RAM and lots of CPUs, one
other option is to run multiple JVMs per physical node that's
sometimes helpful.
One other possibility. In Solr 7.5, you have a ton of metrics
available. If you hit the admin/metrics end point you'll see 150-200
available metrics. Apart from running a profiler to see what's
consuming the most cycles, the metrics can give you a view into what
Solr is doing and may help you pinpoint what's using the most cycles.
Best,
Erick
On Fri, Oct 26, 2018 at 12:23 PM Toke Eskildsen <t...@kb.dk> wrote:
David Hastings <hastings.recurs...@gmail.com> wrote:
Would adding the docValues in the schema, but not reindexing, cause
errors? IE, only apply the doc values after the next reindex, but in
the
meantime keep functioning as there were none until then?
As soon as you specify in the schema that a field has docValues=true,
Solr treats all existing documents as having docValues enabled for that
field. As there is no docValue content, DocValues-aware functionality such
as sorting and faceting will not work for that field, until the documents
has been re-indexed.
- Toke Eskildsen
--
Email Signature
*Sofiia Strochyk
*
s...@interlogic.com.ua <mailto:s...@interlogic.com.ua>
InterLogic
www.interlogic.com.ua <https://www.interlogic.com.ua>
Facebook icon <https://www.facebook.com/InterLogicOfficial> LinkedIn
icon <https://www.linkedin.com/company/interlogic>
