org.apache.lucene.util.fst.FST taking up lot of Java Heap Memory

2020-08-03 Thread sanjay dutt
so that it take less memory. Can someone please give me any pointer that from where Should I start looking to debug the above issue.  Thanks and Regards,Sanjay Dutt Sent from Yahoo Mail on Android

Sort on internal lucene ID led to OOM

2020-08-04 Thread sanjay dutt
Hello, We were investigating one HEAP DUMP in which FielsCaceImpl has occupied around 5GB. Upon further investigation, I got to know that FieldCacheImpl$SortedDocValues occupies 90% of the memory where FieldCacheImpl$CacheKey is "id". When I checked the logs I was trying to find any query in

Re: org.apache.lucene.util.fst.FST taking up lot of Java Heap Memory

2020-08-06 Thread sanjay dutt
and Regards,Sanjay Dutt On Thursday, August 6, 2020, 12:10:04 AM GMT+5:30, David Smiley wrote: What is the Solr field type definition for this field?  And what sort of spatial data do you add here -- just points or what? ~ David Smiley Apache Lucene/Solr Search Developer http

Re: Solr Down Issue

2020-08-09 Thread sanjay dutt
It could be OOM killer. Try to monitor it's heap and there is a script in bin which basically kills solr when OOM occurs. Sent from Yahoo Mail on Android On Sun, Aug 9, 2020 at 8:14 PM, Ben wrote: Can you send solr logs? Best, Ben On Sun, Aug 9, 2020, 9:55 AM Rashmi Jain wrote: >

Re: org.apache.lucene.util.fst.FST taking up lot of Java Heap Memory

2020-08-07 Thread sanjay dutt
to having the FSTs off-heap -- at least the ones associated with the field indexes. ~ David Smiley Apache Lucene/Solr Search Developer http://www.linkedin.com/in/davidwsmiley On Thu, Aug 6, 2020 at 8:19 PM sanjay dutt wrote: > FieldType defined with class solr.SpatialRecursivePrefixTreeFieldType &

SPLITSHARD failed after running for hours

2020-08-12 Thread sanjay dutt
Hello Solr community, We tried to split shard of one collection which contains 80M documents. After running for few hours it failed with the exception org.apache.solr.common.SolrException. Upon further investigation, I found below exception Caused by:

Re: Sort on internal lucene ID led to OOM

2020-08-05 Thread sanjay dutt
not doing what you expect. Best, Erick > On Aug 4, 2020, at 10:19 PM, sanjay dutt > wrote: > > Hello, > We were investigating one HEAP DUMP in which FielsCaceImpl has occupied > around 5GB. Upon further investigation, I got to know that > FieldCacheImpl$SortedDocValues occup

Re: _version_ taking too much memory

2020-11-18 Thread sanjay dutt
Thank you so much Shalin. Yes that would reduce the memory requirement at run time.But my next question, Is the "firstSearcher" doing this I mean due to which it tries to find the "Max value of version" and for that it it has to do the un-inversion.  Thanks and

_version_ taking too much memory

2020-11-17 Thread sanjay dutt
Solr Collection setup Shards :- 2Replication :- 4Documents :- 569 Million (I know it's too much)Heap Space :- 12GB So basically, above collection is having OutOfMemory issues. And upon inspection, I got to know that  org.apache.lucene.uninverting.FieldCacheImpl$LongsFromArray for field