Re: IndexWriter, DirectoryTaxonomyWriter & SearcherTaxonomyManager synchronization

2016-09-28 Thread William Moss
Confusingly, I can't for the life of me seem to replicate this error (it used to be quite easy). I wish I still had the logs somewhere :-( One important thing I left out is that we're also calling delete a lot. Most of the time when we get an update from Kafka it's an update to an existing documen

Re: IndexWriter, DirectoryTaxonomyWriter & SearcherTaxonomyManager synchronization

2016-09-28 Thread Shai Erera
*> However, that should not lead to NSFE. At worst it should lead to> "ordinal is not known" (maybe as an AIOOBE) from the taxonomy reader.* That is correct, this interleaving indexing case can potentially result in an AIOOBE like exception during faceted search, when the facets that are in the "

Re: IndexWriter, DirectoryTaxonomyWriter & SearcherTaxonomyManager synchronization

2016-09-28 Thread Michael McCandless
On Wed, Sep 28, 2016 at 3:05 AM, William Moss wrote: > Thank you both for your quick reply! You're welcome! > * We actually tried the upgrade to 6.0 a few months back (when that was the > newest) and were getting similar errors to the ones I'm seeing now. We were > not able to track them down, w

Re: IndexWriter, DirectoryTaxonomyWriter & SearcherTaxonomyManager synchronization

2016-09-28 Thread Michael McCandless
On Tue, Sep 27, 2016 at 7:05 AM, Shai Erera wrote: > Hmm ... the commit part of the two indexes is always tricky. The javadocs > are correct because the order of indexing is as follows: when you index a > document with facets, the facets are first added to the taxonomy index and > only then the do

Re: Clarification Regarding TieredMergePolicy

2016-09-28 Thread Michael McCandless
See TMP.setMaxMergedSegmentMB, which is the (target) maximum sized segment produced as a result by a merge. Mike McCandless http://blog.mikemccandless.com On Tue, Sep 27, 2016 at 5:39 PM, aravinth thangasami wrote: > what decides that the segment is too big ??? > I hope you don't mean the MAX_

Re: Research problems on numeric values into text (with. or,)

2016-09-28 Thread Michael McCandless
Thank you for bringing closure. Mike McCandless http://blog.mikemccandless.com On Wed, Sep 28, 2016 at 11:56 AM, Jérémy GUYENOT wrote: > Hi Michael, > > > > I just find my problem. Du to a Lucene problem that it index “abcd.” Like > a word we added into our code a regex to add space between “a

RE: Research problems on numeric values into text (with. or,)

2016-09-28 Thread Jérémy GUYENOT
Hi Michael, I just find my problem. Du to a Lucene problem that it index “abcd.” Like a word we added into our code a regex to add space between “abcd” and “.” (or punctuation caracters). So I update this regex and it wxorks fine. The code before: // Add space between word and punctuation cara