Re: enhancement for SynonymFilter

2016-11-18 Thread Michael McCandless
Hmm I didn't realize there was that change in behavior between versions. But, in 6.3.0, can't you look for a token of type SYNONYM whose posInc=0 and then know that the previous (posInc>0) token had caused that synonym? You just need a bit of caching, until all synonyms for a given token have bee

Re: enhancement for SynonymFilter

2016-11-18 Thread Bernd Fehling
Am 18.11.2016 um 08:58 schrieb Bernd Fehling: > Hi Mike, > > let me explain. > > First, after looking deeper inside I noticed that the Filters are used > like a stack and called backwards. So the first incrementToken goes > to the last filter in the chain. That one also uses incrementToken and

Re: enhancement for SynonymFilter

2016-11-18 Thread Bernd Fehling
Hi Mike, let me explain. First, after looking deeper inside I noticed that the Filters are used like a stack and called backwards. So the first incrementToken goes to the last filter in the chain. That one also uses incrementToken and and calls its predecessor in the chain and so on. So everythin

Re: enhancement for SynonymFilter

2016-11-17 Thread Michael McCandless
Hmm are you saying SynonymFilter in 4.10.4 has this capability but 6.3.0 lost it? So you you have a synonym "wow that's funny" -> "wtf", you want the token for "wow" to state that it has a synonym? Using the PositionLengthAttribute you should be able to reconstruct this, because when you see "wtf

enhancement for SynonymFilter

2016-11-17 Thread Bernd Fehling
Currently I'm tackling a problem with SynonymFilter while going from 4.10.4 to 6.3.0. For a special solution I need to know if a word (or multiword) is producing synonyms in SynonymFilter. Therefore I suggest the enhancement of "hasSynonyms" for SynonymFilter. A workaroud would be to buffer all