@Erick Erickson<mailto:erickerick...@gmail.com> and @Mikhail Khludnev
got it, the explanation is very clear. Thank you for your help. ________________________________ From: Hongxu Ma <inte...@outlook.com> Sent: Tuesday, February 18, 2020 10:22 To: Vadim Ivanov <vadim.iva...@spb.ntk-intourist.ru>; solr-user@lucene.apache.org <solr-user@lucene.apache.org> Subject: Re: A question about solr filter cache Thank you @Vadim Ivanov<mailto:vadim.iva...@spb.ntk-intourist.ru> I know that admin page, but I cannot find the memory usage of filter cache (only has "CACHE.searcher.filterCache.size", I think it's the used slot number of filtercache) There is my output (solr version 7.3.1): filterCache * class: org.apache.solr.search.FastLRUCache * description: Concurrent LRU Cache(maxSize=512, initialSize=512, minSize=460, acceptableSize=486, cleanupThread=false) * stats: * CACHE.searcher.filterCache.cumulative_evictions: 0 * CACHE.searcher.filterCache.cumulative_hitratio: 0.5 * CACHE.searcher.filterCache.cumulative_hits: 1 * CACHE.searcher.filterCache.cumulative_inserts: 1 * CACHE.searcher.filterCache.cumulative_lookups: 2 * CACHE.searcher.filterCache.evictions: 0 * CACHE.searcher.filterCache.hitratio: 0.5 * CACHE.searcher.filterCache.hits: 1 * CACHE.searcher.filterCache.inserts: 1 * CACHE.searcher.filterCache.lookups: 2 * CACHE.searcher.filterCache.size: 1 * CACHE.searcher.filterCache.warmupTime: 0 ________________________________ From: Vadim Ivanov <vadim.iva...@spb.ntk-intourist.ru> Sent: Monday, February 17, 2020 17:51 To: solr-user@lucene.apache.org <solr-user@lucene.apache.org> Subject: RE: A question about solr filter cache You can easily check amount of RAM used by core filterCache in Admin UI: Choose core - Plugins/Stats - Cache - filterCache It shows useful information on configuration, statistics and current RAM usage by filter cache, as well as some examples of current filtercaches in RAM Core, for ex, with 10 mln docs uses 1.3 MB of Ram for every filterCache > -----Original Message----- > From: Hongxu Ma [mailto:inte...@outlook.com] > Sent: Monday, February 17, 2020 12:13 PM > To: solr-user@lucene.apache.org > Subject: A question about solr filter cache > > Hi > I want to know the internal of solr filter cache, especially its memory usage. > > I googled some pages: > https://teaspoon-consulting.com/articles/solr-cache-tuning.html > https://lucene.472066.n3.nabble.com/Solr-Filter-Cache-Size-td4120912.html > (Erick Erickson's answer) > > All of them said its structure is: fq => a bitmap (total doc number bits), but I > think it's not so simple, reason: > Given total doc number is 1 billion, each filter cache entry will use nearly > 1GB(1000000000/8 bit), it's too big and very easy to make solr OOM (I have a > 1 billion doc cluster, looks it works well) > > And I also checked solr node, but cannot find the details (only saw using > DocSets structure) > > So far, I guess: > > * degenerate into an doc id array/list when the bitmap is sparse > * using some compressed bitmap, e.g. roaring bitmaps > > which one is correct? or another answer, thanks you very much!
