You're right, the sidecar index does bring some challenges into the picture, but we're using it like that for many years, in distributed mode too, and so far it wasn't an issue. I opened LUCENE-3786 to create SearcherTaxoManager which lets you manage an IndexSearcher and TaxonomyReader pairs, like SearcherManager does. I am thinking maybe this object will also manage the commits to both indexes.
Keeping them in sync is a delicate matter, but certainly doable, even more so, now that IndexWriter lets you commit just commitData. The taxonomy manages the global ordinals for categories. The first version of it used some files (maybe a B-Tree, I don't recall), but moving to a Lucene index was a huge gain. The code became very simple, and we could enjoy Lucene's robustness and commit semantics. The global ordinals are a huge benefit IMO, as they let you do all the work on integers, rather than strings, and allow you to do faceting both off-disk an in-memory. They are also NRT friendly (and DirectoryTaxonomyReader is now NRT too!). I'm not too familiar with Solr adapters .. will Solr NRT, SolrCloud etc. work with any adapter, even one that carries along a sidecar index/data structure? I'm mostly worried about replication, because distributed indexing should not be affected by the existence of the taxonomy index. Shai On Tue, Dec 11, 2012 at 7:40 AM, David Smiley (@MITRE.org) < dsmi...@mitre.org> wrote: > Shai Erera wrote > > Yonik, unlike Solr facets (which manage everything in the search index), > > the Lucene module comes with a sidecar taxonomy index, so e.g. when Solr > > replicates shards, it will need to replicate one other index files. > That's > > the big difference, the rest are miniscule I think. And of course, Solr > > has > > a much higher level API than Lucene, so we'll need to translate those > APIs > > to the facets module. > > Shai, > RE: Sidecar index -- That's a huge difference and a shortcoming; no? Do > you somehow take care to avoid a stale view on the sidecar index during a > commit? > > On the upside; if this does proper hierarchical faceting then a Solr > adapter > for it would be awesome. > > ~ David > > > > ----- > Author: > http://www.packtpub.com/apache-solr-3-enterprise-search-server/book > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Solr-faceting-vs-Lucene-faceting-tp4025577p4025928.html > Sent from the Lucene - Java Developer mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > >
