I agree. I'll add this information to the wiki. On 9 June 2010 14:32, Jean-Sebastien Vachon <js.vac...@videotron.ca> wrote: > ok great. > > I believe this should be mentioned in the wiki. > > Later > > On 2010-06-09, at 4:06 AM, Martijn v Groningen wrote: > >> The fieldCollapseCache should not be used as it is now, it uses too >> much memory. It stores any information relevant for a field collapse >> search. Like document collapse counts, collapsed document ids / >> fields, collapsed docset and uncollapsed docset (everything per unique >> search). So the memory usage will grow for each unique query (and fast >> with all this information). So its best I think to disable this cache >> for now. >> >> Martijn >> >> On 8 June 2010 19:05, Jean-Sebastien Vachon <js.vac...@videotron.ca> wrote: >>> Hi All, >>> >>> I've been running some tests using 6 shards each one containing about 1 >>> millions documents. >>> Each shard is running in its own virtual machine with 7 GB of ram (5GB >>> allocated to the JVM). >>> After about 1100 unique queries the shards start to struggle and run out of >>> memory. I've reduced all >>> other caches without significant impact. >>> >>> When I remove completely the fieldCollapseCache, the server can keep up for >>> hours >>> and use only 2 GB of ram. (I'm even considering returning to a 32 bits JVM) >>> >>> The size of the fieldCollapseCache was set to 5000 items. How can 5000 >>> items eat 3 GB of ram? >>> >>> Can someone tell me what is put in this cache? Has anyone experienced this >>> kind of problem? >>> >>> I am running Solr 1.4.1 with patch 236. All requests are collapsing on a >>> single field (pint) and >>> collapse.maxdocs set to 200 000. >>> >>> Thanks for any hints... >>> >>> > >
-- Met vriendelijke groet, Martijn van Groningen
