TokenizerChain ClassCastException
---------------------------------

                 Key: SOLR-1382
                 URL: https://issues.apache.org/jira/browse/SOLR-1382
             Project: Solr
          Issue Type: Bug
          Components: Analysis
    Affects Versions: 1.4
            Reporter: Simon Glentworth
             Fix For: 1.4


TokenizerChain casts the result of create method in TokenizerFactory to 
Tokenizer.

{noformat}
  public TokenStreamInfo getStream(String fieldName, Reader reader) {
    Tokenizer tk = (Tokenizer)tokenizer.create(charStream(reader));
{noformat}

The TokenizerFactory interface specifies a return type of TokenStream not 
Tokenizer.

{noformat}
  /** Creates a TokenStream of the specified input */
  public TokenStream create(Reader input);
{noformat}

Tokenizer is a subclass of TokenStream and TokenizerChain is luckily working in 
most cases since most token factories return a Tokenizer. The following token 
factories return a TokenStream and will cause a ClassCastException.

HTMLStripWhitespaceTokenizerFactory
HTMLStripStandardTokenizerFactory
PatternTokenizerFactory
TrieTokenizerFactory





-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply via email to