Adding extra information.
Our index size is around 120 GB (2 shard + 2 replica).
We have 400 GB RAM on our windows server. Solr is assigned 50 GB RAM. So
there is huge amount of free RAM (>300 GB) is available for OS.
We have very simple query which returns only 5 solr documents. Under load
condition it takes 100 ms to 2000 ms.
-----Original Message-----
From: Maulin Rathod
Sent: 03 March 2016 12:24
To: solr-user@lucene.apache.org
Subject: RE: Solr Configuration (Caching & RAM) for performance Tuning
we do soft commit when we insert/update document.
//Insert Document
UpdateResponse resp = cloudServer.add(doc, 1000); if (resp.getStatus() == 0) {
success = true;
}
//Update Document
UpdateRequest req = new UpdateRequest(); req.setCommitWithin(1000);
req.add(docs); UpdateResponse resp = req.process(cloudServer); if
(resp.getStatus() == 0) {
success = true;
}
Here is commit settings in solrconfig.xml.
<autoCommit>
<maxTime>600000</maxTime>
<maxDocs>20000</maxDocs>
<openSearcher>false</openSearcher>
</autoCommit>
<autoSoftCommit>
<maxTime>${solr.autoSoftCommit.maxTime:-1}</maxTime>
</autoSoftCommit>
-----Original Message-----
From: Binoy Dalal [mailto:binoydala...@gmail.com]
Sent: 03 March 2016 11:57
To: solr-user@lucene.apache.org
Subject: Re: Solr Configuration (Caching & RAM) for performance Tuning
1) Experiment with the autowarming settings in solrconfig.xml. Since in your
case, you're indexing so frequently consider setting the count to a low number,
so that not a lot of time is spent warming the caches.
Alternatively if you're not very big on initial query response times being
small, you could turn off auto warming all together.
Also how are your commit settings configured?
Do you do a hard commit every 10 seconds or do you have soft committing enabled?
2) 50Gb memory is way to high to assign to just solr and it is unnecessary.
Solr loads your index into the OS cache. The index is not held in the JVM heap.
So it is essential that your OS have enough free memory available to load the
entire index.
Since you're only seeing about a 2gb use of your JVM memory, set your heap size
to something around 4gbs.
Also, how big is your index?
On Thu, 3 Mar 2016, 11:39 Maulin Rathod, <mrat...@asite.com> wrote:
> Hi,
>
> We are using Solr 5.2 (on windows 2012 server/jdk 1.8) for document
> content indexing/querying. We found that querying slows down
> intermittently under load condition.
>
> In our analysis we found two issues.
>
> 1) Solr is not effectively using caching.
>
> Whenever new document indexed, it opens new searcher and hence cache
> will become invalid (as cache was associated with old Index Searcher).
> In our scenario, new documents are indexed very frequently (at least
> 10 document are indexed per minute). So effectively cache will not be
> useful as it will open new searcher frequently to make new documents
> available for searching.
> How can improve caching usage?
>
>
> 2) RAM is not utilized
>
> We observed that Solr is using only 1-2 GB of heap even though we have
> assign 50 GB. Seems like it is not loading index into RAM which leads
> to high IO. Is it possible to configure Solr to fully load indexes in memory?
> Don't find any documentation about this. How can we increase RAM usage
> to improve Solr performance?
>
>
> Regards,
>
> Maulin
>
> [CC Award Winners 2015]
>
> --
Regards,
Binoy Dalal
