On Mon, Aug 10, 2009 at 22:50, Grant Ingersoll<gsing...@apache.org> wrote: > > On Aug 10, 2009, at 2:00 PM, Earwin Burrfoot wrote: > >> I'll deviate from the topic somewhat. >> What are exact benefits that new tokenstream API yields? Are we sure >> we want it released with 2.9? >> By now I only see various elaborate problems, but haven't seen a >> single piece of code becoming simpler. > > In theory, it sets up for more indexing/searching possibilities in 3.0, but > in the meantime, it is proving to be quite problematic due to back > compatibility restrictions. I'm not quite sure which exact indexing/searching possibilities does the new API open for us. Some new ways of handling text? Okay, I'd like each token to have one more number in addition to posIncr, so I can have my 'true multiword synonyms'. Maybe, just maybe, there will be a pair of other extensions. Usecases here are really scarce. Plus, if they're successful/useful, they will most probably be included out of the box, so we don't need much flexibility here. Something other than text? Numbers, with good rangequeries. Dates. Spatial data. Your-type-here. For these, flexible text-processing stream-oriented API is totally useless.
> I have serious doubts about releasing this new API until these performance > issues are resolved and better proven out from a usability standpoint. > It simply is too much to swallow for most users, as > Analyzers/TokenStreams/etc. are easily the most common place for people to > inject their own capabilities and there is no way we should be > taking a 30% hit in performance for some theoretical speed up and new search > capability 1 year from now. I have a feeling that best idea, before more damage is done, is to rollback this new API, store the patch, and try rolling it out once again, when we have usecases/more code to justify it. -- Kirill Zakharenko/Кирилл Захаренко (ear...@gmail.com) Home / Mobile: +7 (495) 683-567-4 / +7 (903) 5-888-423 ICQ: 104465785 --------------------------------------------------------------------- To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org For additional commands, e-mail: java-dev-h...@lucene.apache.org
