Re: # open files with SolrCloud
On Sat, Apr 21, 2012 at 9:57 PM, Yonik Seeley yo...@lucidimagination.com wrote: I can reproduce some kind of searcher leak issue here, even w/o SolrCloud, and I've opened https://issues.apache.org/jira/browse/SOLR-3392 With the fix integrated. I do not see the leaking problem anymore with my setup so it seems to be working now. -- Sami Siren
Re: # open files with SolrCloud
Great! I am going to try new Solr 4 build from April 23rd On Sun, Apr 22, 2012 at 11:35 PM, Sami Siren ssi...@gmail.com wrote: On Sat, Apr 21, 2012 at 9:57 PM, Yonik Seeley yo...@lucidimagination.com wrote: I can reproduce some kind of searcher leak issue here, even w/o SolrCloud, and I've opened https://issues.apache.org/jira/browse/SOLR-3392 With the fix integrated. I do not see the leaking problem anymore with my setup so it seems to be working now. -- Sami Siren
Re: # open files with SolrCloud
I can reproduce some kind of searcher leak issue here, even w/o SolrCloud, and I've opened https://issues.apache.org/jira/browse/SOLR-3392 -Yonik lucenerevolution.com - Lucene/Solr Open Source Search Conference. Boston May 7-10
Re: # open files with SolrCloud
Yonik, This same issue we have on our production with Solr 4 Trunk build running on Cent OS, JDK 6 64-bit I have reported java.io.IOException: Map failed and Too many open files issue, i seems their is a search leak in Solr which is not closing them and file being kept open. It would be great help if we can resolve this issue, I was going to try latest build but it seems this issue is not resolved yet http://lucene.472066.n3.nabble.com/Large-Index-and-OutOfMemoryError-Map-failed-td3872891.html On Sat, Apr 21, 2012 at 11:57 AM, Yonik Seeley yo...@lucidimagination.comwrote: I can reproduce some kind of searcher leak issue here, even w/o SolrCloud, and I've opened https://issues.apache.org/jira/browse/SOLR-3392 -Yonik lucenerevolution.com - Lucene/Solr Open Source Search Conference. Boston May 7-10
Re: # open files with SolrCloud
forgot to mention we are not using Solr Cloud yet but we use Lucene NRT feature, This issue is happening WITHOUT Solr Cloud On Sat, Apr 21, 2012 at 8:14 PM, Gopal Patwa gopalpa...@gmail.com wrote: Yonik, This same issue we have on our production with Solr 4 Trunk build running on Cent OS, JDK 6 64-bit I have reported java.io.IOException: Map failed and Too many open files issue, i seems their is a search leak in Solr which is not closing them and file being kept open. It would be great help if we can resolve this issue, I was going to try latest build but it seems this issue is not resolved yet http://lucene.472066.n3.nabble.com/Large-Index-and-OutOfMemoryError-Map-failed-td3872891.html On Sat, Apr 21, 2012 at 11:57 AM, Yonik Seeley yo...@lucidimagination.com wrote: I can reproduce some kind of searcher leak issue here, even w/o SolrCloud, and I've opened https://issues.apache.org/jira/browse/SOLR-3392 -Yonik lucenerevolution.com - Lucene/Solr Open Source Search Conference. Boston May 7-10
Re: # open files with SolrCloud
On Thu, Apr 19, 2012 at 3:12 PM, Sami Siren ssi...@gmail.com wrote: I have a simple solrcloud setup from trunk with default configs; 1 shard with one replica. As few other people have reported there seems to be some kind of leak somewhere that causes the number of open files to grow over time when doing indexing. One thing that correlates with the open file count that the jvm reports is the count of deleted files that solr still keeps open (not sure if the problem is this or something else). The deleted but not closed files are all ending with nrm.cfs, for example solr_home/solr/data/index/_jwk_nrm.cfs (deleted) Any ideas about what could be the cause for this? I don't even know where to start looking... I think the problem is not just the (deleted) files that counts towards the max open files the user is allowed to have. There's also the mmapped files that are kept open and count towards the limit configured in /proc/sys/vm/max_map_count and eventually lead to exceptions like java.io.IOException: Map failed there's plenty of those too visible to lsof (all kinds of lucene index files), for example: java32624 sam DELREG8,36425004 /home/sam/example/solr/data/index/_4.fdx java32624 sam DELREG8,36424981 /home/sam/example/solr/data/index/_4.fdt java32624 sam DELREG8,36425019 /home/sam/example/solr/data/index/_4_nrm.cfs java32624 sam DELREG8,36425016 /home/sam/example/solr/data/index/_4_0.tim java32624 sam DELREG8,36425015 /home/sam/example/solr/data/index/_4_0.prx java32624 sam DELREG8,36425014 /home/sam/example/solr/data/index/_4_0.frq Why are all these files, regular and mmapped kept open even if merging has occurred in the background and the files are deleted? - Sami Siren
# open files with SolrCloud
I have a simple solrcloud setup from trunk with default configs; 1 shard with one replica. As few other people have reported there seems to be some kind of leak somewhere that causes the number of open files to grow over time when doing indexing. One thing that correlates with the open file count that the jvm reports is the count of deleted files that solr still keeps open (not sure if the problem is this or something else). The deleted but not closed files are all ending with nrm.cfs, for example solr_home/solr/data/index/_jwk_nrm.cfs (deleted) Any ideas about what could be the cause for this? I don't even know where to start looking... -- Sami Siren
