[ 
https://issues.apache.org/jira/browse/SOLR-1657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12793002#action_12793002
 ] 

Robert Muir commented on SOLR-1657:
-----------------------------------

Hi Uwe, I was thinking about some ideas on how we could do this, here is what I 
think in general:

* CommonGrams[Query]Filter and RemoveDuplicatesFilter should be implemented 
with new api directly, I think they can be a lot more efficient.
* BufferedTokenStream should be deprecated, it cannot properly support the 
attributes api, only those in Token.
* Non-final tokenstreams should be made final.

So according to the above there are some breaks in api compatibility, but it is 
not possible to move to the new api without breaks. This is because you cannot 
properly support the attributes api, yet at the same time extend 
BufferedTokenStream (see CommonGrams for an example). on top of this these are 
non-final! So it is impossible in my opinion.


> convert the rest of solr to use the new tokenstream API
> -------------------------------------------------------
>
>                 Key: SOLR-1657
>                 URL: https://issues.apache.org/jira/browse/SOLR-1657
>             Project: Solr
>          Issue Type: Task
>            Reporter: Robert Muir
>
> org.apache.solr.analysis:
> BufferedTokenStream
>  -> CommonGramsFilter
>  -> CommonGramsQueryFilter
>  -> RemoveDuplicatesTokenFilter
> CapitalizationFilterFactory
> HyphenatedWordsFilter
> LengthFilter (deprecated, remove)
> PatternTokenizerFactory (remove deprecated methods)
> SynonymFilter
> SynonymFilterFactory
> WordDelimiterFilter
> org.apache.solr.handler:
> AnalysisRequestHandler
> AnalysisRequestHandlerBase
> org.apache.solr.handler.component:
> QueryElevationComponent
> SpellCheckComponent
> org.apache.solr.highlight:
> DefaultSolrHighlighter
> org.apache.solr.search:
> FieldQParserPlugin
> org.apache.solr.spelling:
> SpellingQueryConverter

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply via email to