On May 10, 2006, at 4:11 PM, Yonik Seeley wrote:
Unless you really need special cache behavior, can you use the
LRUCache that comes with Solr?
Sure, I suppose I could use that, but it had more bells and whistles
than I need. I like to look at the interfaces and work from there
and then use base classes that fit. LRUCache didn't seem to fit what
I wanted without contorting its configuration.
It will make your life easier using a
well tested cache implementation.
My cache is really just a static cache of BitSet's for a fixed set of
fields and their values. With my current index size, creating the
cache is incredibly fast (a second or so), but the index will grow
much larger.
You shouldn't really need to implement your own Listener either (while
valid, it's a tougher approach). You could just send your plugin a
message via the builtin QuerytSenderListener.
<listener event="firstSearcher" class="solr.QuerySenderListener">
<arr name="queries">
<lst> <str name="q">fast_warm</str> <str name="start">0</
str> <str name=
"rows">10</str> </lst>
</arr>
</listener>
I see.... which is basically what I was already doing by having the
cache lazy initialize on the first request in the request handler
(aka plugin), except the first request is coming from startup thanks
to the listener architecture.
If you do choose to implement your own listener, you need to register
it, like above.
I did, but omitted it from my previous details:
<listener event="firstSearcher" class="org.nines.CacheFacetsListener"/>
Details on how your facet cache is supposed to work might help with
answering future questions.
For a fixed set of fields (currently 4 or so of them) I'm building a
HashMap keyed by field name, with the values of each key also a
HashMap, keyed by term value. The value of the inner HashMap is a
BitSet representing all documents that have that value for that
field. These BitSets are used for a faceted browser and ANDed
together based on user criteria, as well as combined with full-text
queries using QueryFilter's BitSet. Nothing fancy, and perhaps
something Solr already helps provide?
The question still remains - why isn't my cache available from a
firstSearcher .newSearcher() method? The cache is created prior (as
noted in the console output).
Erik
-Yonik
On 5/10/06, Erik Hatcher <[EMAIL PROTECTED]> wrote:
On May 10, 2006, at 3:01 PM, Yonik Seeley wrote:
> On 5/10/06, Erik Hatcher <[EMAIL PROTECTED]> wrote:
>> I've started down this route, but I'm not sure how to
initialize my
>> cache the first time.
>>
>> I need access to the IndexReader to build the cache, and at this
>> point I don't need any incremental cache updates - if a new
>> IndexSearcher is swapped in, I want to rebuild the cache.
>>
>> Should I combine a custom SolrCache with a newSearcher listener to
>> have it generated right away? I put in a dummy cache and
>> regenerator, but only see the cache .init() method being
called, not
>> the warm() method (on application server startup). How can I
>> bootstrap it such that my cache gets built on app. startup?
>
> If your cache is populated as the result of any request to your
> plugin, simply send a request via a firstSearcher hook. If it's
not
> populated for any request, then send a special request that your
> plugin would recognize as a "populate cache" request.
Sorry I'm being dense today, though I really do appreciate the
incredibly fast response time you and Hoss have on this. My cache is
not available in newSearcher() at startup time:
public class CacheFacetsListener implements SolrEventListener {
public void init(NamedList namedList) {
}
public void postCommit() {
throw new UnsupportedOperationException();
}
public void newSearcher(SolrIndexSearcher newSearcher,
SolrIndexSearcher currentSearcher) {
try {
SolrCache cache = newSearcher.getCache("facet_cache");
if (cache == null) {
System.out.println("!!!!! cache is null");
}
cache.warm(newSearcher, null);
} catch (IOException e) {
log.severe(e.getMessage());
}
}
}
I'm getting the "cache is null" message. Though the cache is created
and init()'d as I see it's diagnostic output in Jetty's console
before the NPE:
public class FacetCache implements SolrCache {
private Map facetCache; // key is field, and key to inner map is
value
private State state;
private void loadFacets(IndexReader reader) throws IOException {
System.out.println("Loading facets for " + reader.numDocs() + "
documents ...");
// ....
System.out.println("Done loading facets.");
}
public Object init(Map args, Object persistence, CacheRegenerator
regenerator) {
state=State.CREATED;
System.out.println("<<<<< FacetCache.init >>>>>");
return persistence;
}
// ......
}
And from solrconfig.xml:
<cache name="facet_cache"
class="org.nines.FacetCache"
/>
The console has this output:
May 10, 2006 3:44:26 PM org.apache.solr.search.SolrIndexSearcher
<init>
INFO: Opening [EMAIL PROTECTED] main
<<<<< FacetCache.init >>>>>
May 10, 2006 3:44:26 PM org.apache.solr.core.SolrCore
registerSearcher
INFO: Registered new searcher [EMAIL PROTECTED] main
!!!!! cache is null
May 10, 2006 3:44:26 PM org.apache.solr.core.SolrException log
SEVERE: java.lang.NullPointerException
at org.nines.CacheFacetsListener.newSearcher
(CacheFacetsListener.java:24)
at org.apache.solr.core.SolrCore$2.call(SolrCore.java:427)
I'm probably making this more difficult than it needs to be, but
today I'm slow :) What am I doing wrong?
Thanks,
Erik
