On 5/10/06, Erik Hatcher <[EMAIL PROTECTED]> wrote:
For a fixed set of fields (currently 4 or so of them) I'm building a HashMap keyed by field name, with the values of each key also a HashMap, keyed by term value. The value of the inner HashMap is a BitSet representing all documents that have that value for that field. These BitSets are used for a faceted browser and ANDed together based on user criteria, as well as combined with full-text queries using QueryFilter's BitSet. Nothing fancy, and perhaps something Solr already helps provide?
Using Solr's DocSet implementations will dramatically speed up your faceted browsing and reduce your memory footprint. You could store these DocSets yourself (and turn off the filter cache so things aren't doubly stored), but here is how I might go about it: In your custom cache, just store the terms for the faceting fields (everything but the bitsets). field1 -> [term1, term2, term3, term4] field2 -> [terma, termb, termc, termd] Then when it comes time to get the count of items matching query x, do count1 = searcher.numDocs(x,TermQuery(term1)) count2 = searcher.numDocs(x,TermQuery(term2)) ... Solr will check the filter cache for "x" and for the TermQuery facets, and generate them on the fly if they are not found. What you loose: - teeny bit of performance because each facet gets looked up in a HashMap (I've profiled... this has been negligible for us) What you gain: - re-use of the filtercache (including the filter for the base query), much faster intersections with less average memory usage & less garbage produced - an ability to easily cap the number of filters used for the facets, allowing a gradual reduction in performance as cache hits lower, rather than an OOM.
The question still remains - why isn't my cache available from a firstSearcher .newSearcher() method? The cache is created prior (as noted in the console output).
Great question... it should be, as it's created and registered in the SolrIndexSearcher constructor. is the cache .name() returning the right thing? -Yonik
