Hi There are a couple of ways you can address that:
Not create an index per-thread, but rather update the global index by all threads. IndexWriter and TaxoWriter support multiple threads. -- Or, if you need to build an index per-thread -- Use a single TaxonomyWriter instance and share between all the threads. TaxoWriter is thread-safe, and that way you can build a single taxonomy index and later use IW.addIndexes. -- Or, if you cannot share TW instance between threads -- Have each thread create its own taxonomy index, but then when you call addIndexes, you need to do two things: - Create a new TW instance and call addTaxonomy on it. - Call IW.addIndexes() with an OrdinalMappingAtomicReader. Look at its jdocs for an example code. Let me know if that works for you. Shai On Wed, Jul 3, 2013 at 6:14 PM, Peng Gao <p...@esri.com> wrote: > Hi Shai, > Thanks for the reply. > Yes I used a single TaxonomyReader instance. > I am adding facets to an existing app, which maintains two indexes, one > for indexing > system tools, and the other indexing user data in folders. > The system tool index contains docs for describing the tool usage, and > etc, which needs to be its own > index. > > It turned out that my problem is not MultiReader. The problem is the > index, i.e. the way it's created. > The app crawls folders in multiple threads, and each thread creates a temp > index. > The main thread merges the temp indexes into the master index, using > IndexWriter.AddIndexes(). > If the temp index has facet index, this approach creates a bad index. > > Is there a way I can build faceted index in multiple threads? > > - Gao Peng > > > -----Original Message----- > > From: Shai Erera [mailto:ser...@gmail.com] > > Sent: Monday, July 01, 2013 8:25 PM > > To: java-user@lucene.apache.org > > Subject: Re: Accumulating facets over a MultiReader > > > > Hi, > > > > I assume that you use a single TaxonomyReader instance? It must be the > same > > for both indexes, that is, both indexes must share the same taxonomy > index, > > or otherwise their ordinals would not match as well as you may hit such > > exceptions since one index may have bigger ordinals than what the > taxonomy > > reader knows about. > > > > Can you share a little bit about your scenario and why do you need to > use a > > MultiReader? > > > > Shai > > > > > > > > On Tue, Jul 2, 2013 at 3:31 AM, Peng Gao <p...@esri.com> wrote: > > > > > How do I accumulate counts over a MultiReader (2 IndexReader)? > > > The following code causes an IOException: > > > > > > ArrayList<FacetRequest> facetRequests = new > > > ArrayList<FacetRequest>(); > > > for (String groupField : groupFields) > > > facetRequests.add(new CountFacetRequest(new > > > CategoryPath(groupField, '/'), 1)); > > > > > > FacetSearchParams facetSearchParams = new > > > FacetSearchParams(facetRequests); > > > StandardFacetsAccumulator accumulator = new > > > StandardFacetsAccumulator(facetSearchParams, reader, taxonomyReader); > > > FacetsCollector facetsCollector = > > > FacetsCollector.create(accumulator); > > > > > > // perform documents search and facets accumulation > > > searcher.search(query, facetsCollector); > > > > > > // return facets results in a proper format > > > return getFacetResults(facetsCollector, sr); > > > > > > > > > Here reader is a MultiReader of 2. I am using Lucene 4.3.1. > > > > > > The following is the callstack. It looks like it has something to do > > > with the MultiReader. > > > How do I make it work? > > > > > > > > > java.io.IOException: PANIC: Got unexpected exception while trying to > > > get/calculate total counts > > > at > > > > > > org.apache.lucene.facet.search.StandardFacetsAccumulator.accumulate(Standar > > dFacetsAccumulator.java:156) > > > at > > > > > > org.apache.lucene.facet.search.StandardFacetsAccumulator.accumulate(Standar > > dFacetsAccumulator.java:378) > > > at > > > > > > org.apache.lucene.facet.search.FacetsCollector.getFacetResults(FacetsCollec > > tor.java:214) > > > at > > > > > > com.esri.arcgis.search.SearchHandler.getFacetResults(SearchHandler.java:551 > > ) > > > at > > > com.esri.arcgis.search.SearchHandler.search(SearchHandler.java:350) > > > at > > > com.esri.arcgis.search.SearchHandler.search(SearchHandler.java:239) > > > at > > > com.esri.arcgis.search.test.Searcher.invokeSearch(Searcher.java:58) > > > at com.esri.arcgis.search.test.Searcher.main(Searcher.java:32) > > > Caused by: java.lang.ArrayIndexOutOfBoundsException: 34 > > > at > > > > > > org.apache.lucene.facet.search.CountingAggregator.aggregate(CountingAggrega > > tor.java:43) > > > at > > > > > > org.apache.lucene.facet.search.StandardFacetsAccumulator.fillArraysForParti > > tion(StandardFacetsAccumulator.java:309) > > > at > > > > > > org.apache.lucene.facet.search.StandardFacetsAccumulator.accumulate(Standar > > dFacetsAccumulator.java:168) > > > at > > > > > > org.apache.lucene.facet.complements.TotalFacetCounts.compute(TotalFacetCoun > > ts.java:176) > > > at > > > > > > org.apache.lucene.facet.complements.TotalFacetCountsCache.computeAndCache(T > > otalFacetCountsCache.java:157) > > > at > > > > > > org.apache.lucene.facet.complements.TotalFacetCountsCache.getTotalCounts(To > > talFacetCountsCache.java:104) > > > at > > > > > > org.apache.lucene.facet.search.StandardFacetsAccumulator.accumulate(Standar > > dFacetsAccumulator.java:129) > > > ... 7 more > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org > >
