Hi,
I am getting IllegalStateException even though I am following the
TokenStream guidelines:
public List<String> tokenize(String text){
List<String> tokens = new ArrayList<>();
try (TokenStream ts = analyzer.tokenStream("",text)){ //note
that this
is implicit finally with close
ts.reset();
while (ts.incrementToken()) {
tokens.add(ts.getAttribute(CharTermAttribute.class).toString());
}
ts.end();
return tokens;
}catch(Exception e){
throw new RuntimeException(e);
}
}
Here's my analyzer creation code:
TokenizerFactory tokenizerFactory = (TokenizerFactory)
obj(config.getTokenizer());
Tokenizer tokenizer = tokenizerFactory.create();
TokenStream stream = new StandardFilter(tokenizer);
return new Analyzer() {
protected TokenStreamComponents createComponents(String
fieldName) {
return new TokenStreamComponents(tokenizer,
stream);
}
};
--
View this message in context:
http://lucene.472066.n3.nabble.com/java-lang-IllegalStateException-TokenStream-contract-violation-close-call-missing-tp4319109.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
