Bharat, No, you should not give the JVM so much memory. Give it enough to avoid overly frequent GC, but don't steal memory from the OS cache.
Otis -- Sematext is hiring -- http://sematext.com/about/jobs.html?mls Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR ----- Original Message ---- > From: bharath venkatesh <bharathv6.proj...@gmail.com> > To: solr-user@lucene.apache.org > Sent: Sun, November 8, 2009 2:15:00 PM > Subject: Re: tracking solr response time > > Thanks Lance for the clear explanation .. are you saying we should give > solr JVM enough memory so that os cache can optimize disk I/O efficiently .. > that means in our case we have 16 GB index so would it be enough to > allocated solr JVM 20GB memory and rely on the OS cache to optimize disk I/O > i .e cache the index in memory ?? > > > below is stats related to cache > > > *name: * queryResultCache *class: * org.apache.solr.search.LRUCache * > version: * 1.0 *description: * LRU Cache(maxSize=512, initialSize=512, > autowarmCount=256, > regenerator=org.apache.solr.search.solrindexsearche...@67e112b3) > *stats: *lookups > : 0 > hits : 0 > hitratio : 0.00 > inserts : 8 > evictions : 0 > size : 8 > cumulative_lookups : 15 > cumulative_hits : 7 > cumulative_hitratio : 0.46 > cumulative_inserts : 8 > cumulative_evictions : 0 > > > *name: * documentCache *class: * org.apache.solr.search.LRUCache * > version: * 1.0 *description: * LRU Cache(maxSize=512, initialSize=512) * > stats: *lookups : 0 > hits : 0 > hitratio : 0.00 > inserts : 0 > evictions : 0 > size : 0 > cumulative_lookups : 744 > cumulative_hits : 639 > cumulative_hitratio : 0.85 > cumulative_inserts : 105 > cumulative_evictions : 0 > > > *name: * filterCache *class: * org.apache.solr.search.LRUCache > *version: *1.0 > *description: * LRU Cache(maxSize=512, initialSize=512, autowarmCount=256, > regenerator=org.apache.solr.search.solrindexsearche...@1e3dbf67) > *stats: *lookups > : 0 > hits : 0 > hitratio : 0.00 > inserts : 20 > evictions : 0 > size : 12 > cumulative_lookups : 64 > cumulative_hits : 60 > cumulative_hitratio : 0.93 > cumulative_inserts : 12 > cumulative_evictions : 0 > > > hits and hit ratio are zero for ducment cache , filter cache and query > cache .. only commulative hits and hitratio has a non zero numbers .. is > this how it is supposed to be .. or do we to configure it properly ? > > Thanks, > Bharath > > > > > > On Sat, Nov 7, 2009 at 5:47 AM, Lance Norskog wrote: > > > The OS cache is the memory used by the operating system (Linux or > > Windows) to store a cache of the data stored on the disk. The cache is > > usually by block numbers and are not correlated to files. Disk blocks > > that are not used by programs are slowly pruned from the cache. > > > > The operating systems are very good at maintaining this cache. It > > usually better to give the Solr JVM enough memory to run comfortably > > and rely on the OS cache to optimize disk I/O, instead of giving it > > all available ram. > > > > Solr has its own caches for certain data structures, and there are no > > solid guidelines for tuning those. The solr/admin/stats.jsp page shows > > the number of hits & deletes for the caches and most people just > > reload that over & over. > > > > On Fri, Nov 6, 2009 at 3:09 AM, bharath venkatesh > > wrote: > > >>I have to state the obvious: you may really want to upgrade to 1.4 when > > > it's out > > > > > > when would solr 1.4 be released .. is there any beta version available ? > > > > > >>We don't have the details, but a machine with 32 GB RAM and 16 GB index > > > should have the whole index cached by >the OS > > > > > > do we have to configure solr for the index to be cached by OS in a > > > optimised way . how does this caching of index in memory happens ? r > > > there any docs or link which gives details regarding the same > > > > > >>unless something else is consuming the memory or unless something is > > > constantly throwing data out of the OS >cache (e.g. frequent index > > > optimization). > > > > > > what are the factors which would cause constantly throwing data out of > > the > > > OS cache (we are doing index optimization only once in a day during > > > midnight ) > > > > > > > > > Thanks, > > > Bharath > > > > > > > > > > > -- > > Lance Norskog > > goks...@gmail.com > >
