Can you tell me please how many non-tokenized single-valued fields your schema uses, and how many documents? Thanks, Fuad
Rahul R wrote: > > My primary issue is not Out of Memory error at run time. It is memory > leaks: > heap space not being released after doing a force GC also. So after > sometime > as progressively more heap gets utilized, I start running out of > memory.... > The verdict however seems unanimous that there are no known memory leak > issues within Solr. I am still looking at my application to analyse the > problem. Thank you. > > On Thu, Aug 13, 2009 at 10:58 PM, Fuad Efendi <f...@efendi.ca> wrote: > >> Most OutOfMemoryException (if not 100%) happening with SOLR are because >> of >> >> http://lucene.apache.org/java/2_4_0/api/org/apache/lucene/search/FieldCache. >> html >> - it is used internally in Lucene to cache Field value and document ID. >> >> My very long-term observations: SOLR can run without any problems few >> days/months and unpredictable OOM happens just because someone tried >> sorted >> search which will populate array with IDs of ALL documents in the index. >> >> The only solution: calculate exactly amount of RAM needed for >> FieldCache... >> For instance, for 100,000,000 documents single instance of FieldCache may >> require 8*100,000,000 bytes (8 bytes per document ID?) which is almost >> 1Gb >> (at least!) >> >> >> I didn't notice any memory leaks after I started to use 16Gb RAM for SOLR >> instance (almost a year without any restart!) >> >> >> >> >> -----Original Message----- >> From: Rahul R [mailto:rahul.s...@gmail.com] >> Sent: August-13-09 1:25 AM >> To: solr-user@lucene.apache.org >> Subject: Re: JVM Heap utilization & Memory leaks with Solr >> >> *You should try to generate heap dumps and analyze the heap using a tool >> like the Eclipse Memory Analyzer. Maybe it helps spotting a group of >> objects holding a large amount of memory* >> >> The tool that I used also allows to capture heap snap shots. Eclipse had >> a >> lot of pre-requisites. You need to apply some three or five patches >> before >> you can start using it........ My observations with this tool were that >> some >> Hashmaps were taking up a lot of space. Although I could not pin it down >> to >> the exact HashMap. These would either be weblogic's or Solr's.... I will >> anyway give eclipse's a try and see how it goes. Thanks for your input. >> >> Rahul >> >> On Wed, Aug 12, 2009 at 2:15 PM, Gunnar Wagenknecht >> <gun...@wagenknecht.org>wrote: >> >> > Rahul R schrieb: >> > > I tried using a profiling tool - Yourkit. The trial version was free >> for >> > 15 >> > > days. But I couldn't find anything of significance. >> > >> > You should try to generate heap dumps and analyze the heap using a tool >> > like the Eclipse Memory Analyzer. Maybe it helps spotting a group of >> > objects holding a large amount of memory. >> > >> > -Gunnar >> > >> > -- >> > Gunnar Wagenknecht >> > gun...@wagenknecht.org >> > http://wagenknecht.org/ >> > >> > >> >> >> > > -- View this message in context: http://www.nabble.com/JVM-Heap-utilization---Memory-leaks-with-Solr-tp24802380p25017767.html Sent from the Solr - User mailing list archive at Nabble.com.
