Hi Guys I am also seeing this problem.
I am using SOLR 4 from Trunk and seeing this issue repeat every day. Any inputs about how to resolve this would be great -Saroj On Thu, Jul 26, 2012 at 8:33 AM, Karthick Duraisamy Soundararaj < karthick.soundara...@gmail.com> wrote: > Did you find any more clues? I have this problem in my machines as well.. > > On Fri, Jun 29, 2012 at 6:04 AM, Bernd Fehling < > bernd.fehl...@uni-bielefeld.de> wrote: > > > Hi list, > > > > while monitoring my solr 3.6.1 installation I recognized an increase of > > memory usage > > in OldGen JVM heap on my slave. I decided to force Full GC from jvisualvm > > and > > send optimize to the already optimized slave index. Normally this helps > > because > > I have monitored this issue over the past. But not this time. The Full GC > > didn't free any memory. So I decided to take a heap dump and see what > > MemoryAnalyzer > > is showing. The heap dump is about 23 GB in size. > > > > 1.) > > Report Top consumers - Biggest Objects: > > Total: 12.3 GB > > org.apache.lucene.search.FieldCacheImpl : 8.1 GB > > class java.lang.ref.Finalizer : 2.1 GB > > org.apache.solr.util.ConcurrentLRUCache : 1.5 GB > > org.apache.lucene.index.ReadOnlySegmentReader : 622.5 MB > > ... > > > > As you can see, Finalizer has already reached 2.1 GB!!! > > > > * java.util.concurrent.ConcurrentHashMap$Segment[16] @ 0x37b056fd0 > > * segments java.util.concurrent.ConcurrentHashMap @ 0x39b02d268 > > * map org.apache.solr.util.ConcurrentLRUCache @ 0x398f33c30 > > * referent java.lang.ref.Finalizer @ 0x37affa810 > > * next java.lang.ref.Finalizer @ 0x37affa838 > > ... > > > > Seams to be org.apache.solr.util.ConcurrentLRUCache > > The attributes are: > > > > Type | Name | Value > > ----------------------------- > > boolean| isDestroyed | true > > ----------------------------- > > ref | cleanupThread | null > > ---------------------------- > > ref | evictionListener | null > > ------------------------------- > > long | oldestEntry | 0 > > ---------------------------------- > > int | acceptableWaterMark | 9500 > > > -------------------------------------------------------------------------- > > ref | stats | org.apache.solr.util.ConcurrentLRUCache$Stats > > @ 0x37b074dc8 > > ------------------------ > > boolean| islive | true > > ------------------------------------- > > boolean| newThreadForCleanup | false > > ---------------------------- > > boolean| isCleaning | false > > > > > -------------------------------------------------------------------------------- > > ref | markAndSweepLock | java.util.concurrent.locks.ReentrantLock @ > > 0x39bf63978 > > ----------------------------- > > int | lowerWaterMark | 9000 > > ----------------------------- > > int | upperWaterMark | 10000 > > ----------------------------- > > ref | map | java.util.concurrent.ConcurrentHashMap @ > > 0x39b02d268 > > ------------------------------------------------------------------ > > > > > > > > > > 2.) > > While searching for open files and their references I noticed that there > > are references to > > index files which are already deleted from disk. > > E.g. recent index files are "data/index/_2iqw.frq" and > > "data/index/_2iqx.frq". > > But I also see references to "data/index/_2hid.frq" which are quite old > > and are deleted way back > > from earlier replications. > > I have to analyze this a bit deeper. > > > > > > So far my report, I go on analyzing this huge heap dump. > > If you need any other info or even the heap dump, let me know. > > > > > > Regards > > Bernd > > > > >
