[ https://issues.apache.org/jira/browse/LUCENE-4642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13563791#comment-13563791 ]
Uwe Schindler edited comment on LUCENE-4642 at 1/27/13 12:20 PM: ----------------------------------------------------------------- bq. And I guess I was secretly hoping we could remove Tokenizer(AttributeSource) if we fixed the solr hack. This is my opinion, too! To remove the hack I have an idea (but it is also a hack). The main problem is Solr, which cannot work with plain TokenStreams, it always needs a Tokenizer (which is a serious limitation for special field types like numerics). The better hack I have is to write a fake AttributeFactory, that simply returns the attribute implementations of the underlying NumericTokenStream. I will attach a patch. Then we can remove new Tokenizer(AttributeSource), which is horrible and incorrect. was (Author: thetaphi): bq. And I guess I was secretly hoping we could remove Tokenizer(AttributeSource) if we fixed the solr hack. This is my opinion, too! > TokenizerFactory should provide a create method with a given AttributeSource > ---------------------------------------------------------------------------- > > Key: LUCENE-4642 > URL: https://issues.apache.org/jira/browse/LUCENE-4642 > Project: Lucene - Core > Issue Type: Improvement > Components: modules/analysis > Affects Versions: 4.1 > Reporter: Renaud Delbru > Assignee: Steve Rowe > Labels: analysis, attribute, tokenizer > Fix For: 4.2, 5.0 > > Attachments: LUCENE-4642.patch, LUCENE-4642.patch > > > All tokenizer implementations have a constructor that takes a given > AttributeSource as parameter (LUCENE-1826). However, the TokenizerFactory > does not provide an API to create tokenizers with a given AttributeSource. > Side note: There are still a lot of tokenizers that do not provide > constructors that take AttributeSource and AttributeFactory. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org