On Thu, Nov 6, 2014 at 3:41 PM, [email protected] <[email protected]> wrote: > On Thu, Nov 6, 2014 at 3:19 PM, Robert Muir <[email protected]> wrote: >> >> Do the concatenation yourself with your own TokenStream. You can index >> a field with a tokenstream for expert cases (the individual stored >> values can be added separately) > > > Yes, but that’s quite awkward and a fair amount of surrounding code when, in > the end, it could be so much simpler if somehow the TokenStream could be > notified. I’d feel a little better about it if Lucene included the > tokenStream concatenating code (I’ve done a prototype for this, I could work > on it more and contribute) and if the Solr layer had a nice way of > presenting all the values to the Solr FieldType at once instead of > separately — SOLR-4329. > >> >> No need to make the tokenstream API more complicated: its already very >> complicated. > > > Ehh, that’s arguable. Steve’s suggestion amounts to one line of production > code (javadoc & test is separate). If that’s too much then adding a boolean > argument to reset() would feel cleaner, be 0 lines of new code, but would be > backwards-incompatible. Shrug.
Thats just not true, thats why I am against such a change. It is not one line, it makes the "protocol" of tokenstream a lot more complex, we have to ensure the correct values are passed by all consumers (including indexwriter) etc. Same goes regardless of whether it is extra parameters or strange values. Its also bogus to add such stuff when its specific to indexwriter concatenating multiple fields, which anyone can do themselves, with a TokenStream. Its unnecessary. Instead we already provide an expert api (index a TokenStream) for you to do whatever it is you want, without dirtying up lucene's API. Sorry, i dont think we should hack booleans into indexwriter or tokenstream or analyzer for this expert use case, when we already supply you an API to do it, and just "dont wanna". --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
