Re: solr4 performance question

2014-04-08 Thread Erick Erickson
What do you have for hour _softcommit_ settings in solrconfig.xml? I'm guessing you're using SolrJ or similar, but the solrconfig settings will trip a commit as well. For that matter ,what are all our commit settings in solrconfig.xml, both hard and soft? Best, Erick On Tue, Apr 8, 2014 at

Re: solr4 performance question

2014-04-08 Thread Furkan KAMACI
Hi Joshi; Click to the Plugins/Stats section under your collection at Solr Admin UI. You will see the cache statistics for different types of caches. hitratio and evictions are good statistics to look at first. On the other hand you should read here:

RE: solr4 performance question

2014-04-08 Thread Joshi, Shital
-binary @- -H 'Content-type:text/plain; charset=utf-8' EnD) -Original Message- From: Erick Erickson [mailto:erickerick...@gmail.com] Sent: Tuesday, April 08, 2014 2:21 PM To: solr-user@lucene.apache.org Subject: Re: solr4 performance question What do you have for hour _softcommit_

Re: solr4 performance question

2014-04-08 Thread Erick Erickson
/plain; charset=utf-8' EnD) -Original Message- From: Erick Erickson [mailto:erickerick...@gmail.com] Sent: Tuesday, April 08, 2014 2:21 PM To: solr-user@lucene.apache.org Subject: Re: solr4 performance question What do you have for hour _softcommit_ settings in solrconfig.xml? I'm

RE: Solr4 performance

2014-02-28 Thread Joshi, Shital
: Thursday, February 27, 2014 3:45 PM To: solr-user@lucene.apache.org Subject: Re: Solr4 performance You would get more room for disk cache by reducing your large heap. Otherwise, you'd have to add more RAM to your systems or shard your index to more nodes to gain more RAM that way. The Linux VM

RE: Solr4 performance

2014-02-27 Thread Joshi, Shital
Hi Michael, If page cache is the issue, what is the solution? Thanks! -Original Message- From: Michael Della Bitta [mailto:michael.della.bi...@appinions.com] Sent: Monday, February 24, 2014 9:54 PM To: solr-user@lucene.apache.org Subject: Re: Solr4 performance I'm not sure how you're

Re: Solr4 performance

2014-02-27 Thread Shawn Heisey
On 2/27/2014 1:09 PM, Joshi, Shital wrote: If page cache is the issue, what is the solution? What operating system are you using, and what tool are you looking at to see your memory usage? Can you share a screenshot with us? Use a file sharing website for that - the list generally doesn't

Re: Solr4 performance

2014-02-27 Thread Michael Della Bitta
is the solution? Thanks! -Original Message- From: Michael Della Bitta [mailto:michael.della.bi...@appinions.com] Sent: Monday, February 24, 2014 9:54 PM To: solr-user@lucene.apache.org Subject: Re: Solr4 performance I'm not sure how you're measuring free RAM. Maybe this will help: http

RE: Solr4 performance

2014-02-24 Thread Joshi, Shital
? -Original Message- From: Michael Della Bitta [mailto:michael.della.bi...@appinions.com] Sent: Friday, February 21, 2014 5:28 PM To: solr-user@lucene.apache.org Subject: Re: Solr4 performance It could be that your query is churning the page cache on that node sometimes, so Solr pauses so the OS

Re: Solr4 performance

2014-02-24 Thread Michael Della Bitta
[mailto:michael.della.bi...@appinions.com] Sent: Friday, February 21, 2014 5:28 PM To: solr-user@lucene.apache.org Subject: Re: Solr4 performance It could be that your query is churning the page cache on that node sometimes, so Solr pauses so the OS can drag those pages off of disk. Have you tried

RE: Solr4 performance

2014-02-20 Thread Joshi, Shital
:55 PM To: solr-user@lucene.apache.org Subject: Re: Solr4 performance On 2/18/2014 2:14 PM, Joshi, Shital wrote: Thanks much for all suggestions. We're looking into reducing allocated heap size of Solr4 JVM. We're using NRTCachingDirectoryFactory. Does it use MMapDirectory internally? Can

RE: Solr4 performance

2014-02-20 Thread Michael Della Bitta
:55 PM To: solr-user@lucene.apache.org Subject: Re: Solr4 performance On 2/18/2014 2:14 PM, Joshi, Shital wrote: Thanks much for all suggestions. We're looking into reducing allocated heap size of Solr4 JVM. We're using NRTCachingDirectoryFactory. Does it use MMapDirectory internally

RE: Solr4 performance

2014-02-18 Thread Joshi, Shital
for 700 mil documents) Thanks! -Original Message- From: Roman Chyla [mailto:roman.ch...@gmail.com] Sent: Wednesday, February 12, 2014 3:17 PM To: solr-user@lucene.apache.org Subject: Re: Solr4 performance And perhaps one other, but very pertinent, recommendation is: allocate only as little

Re: Solr4 performance

2014-02-18 Thread Shawn Heisey
On 2/18/2014 2:14 PM, Joshi, Shital wrote: Thanks much for all suggestions. We're looking into reducing allocated heap size of Solr4 JVM. We're using NRTCachingDirectoryFactory. Does it use MMapDirectory internally? Can someone please confirm? In Solr, NRTCachingDirectory does indeed use

RE: Solr4 performance

2014-02-12 Thread Joshi, Shital
Does Solr4 load entire index in Memory mapped file? What is the eviction policy of this memory mapped file? Can we control it? _ From: Joshi, Shital [Tech] Sent: Wednesday, February 05, 2014 12:00 PM To: 'solr-user@lucene.apache.org' Subject: Solr4

Re: Solr4 performance

2014-02-12 Thread Shalin Shekhar Mangar
No, Solr doesn't load the entire index in memory. I think you'll find Uwe's blog most helpful on this matter: http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html On Thu, Feb 13, 2014 at 12:27 AM, Joshi, Shital shital.jo...@gs.com wrote: Does Solr4 load entire index in Memory

Re: Solr4 performance

2014-02-12 Thread Greg Walters
Shital, Take a look at http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html as it's a pretty decent explanation of memory mapped files. I don't believe that the default configuration for solr is to use MMapDirectory but even if it does my understanding is that the entire

Re: Solr4 performance

2014-02-12 Thread Shawn Heisey
On 2/12/2014 12:07 PM, Greg Walters wrote: Take a look at http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html as it's a pretty decent explanation of memory mapped files. I don't believe that the default configuration for solr is to use MMapDirectory but even if it does my

Re: Solr4 performance

2014-02-12 Thread Roman Chyla
And perhaps one other, but very pertinent, recommendation is: allocate only as little heap as is necessary. By allocating more, you are working against the OS caching. To know how much is enough is bit tricky, though. Best, roman On Wed, Feb 12, 2014 at 2:56 PM, Shawn Heisey