Hi,
In our project for Lucene migration from 2.4.0 to 8.11.2, we need your
suggestion to address a case.
With Lucene 2.4.0, we were using the kind of below code snippet.
With Lucene 8.11.2[Written snippet below], we need to extract the startOffset &
endOffset value for further some calculation similar to Lucene 2.4.0.
Is there any easy way/API to extract the values from tokenStream?
//Lucene 2.4.0
===========================================================================
Token token;
TokenStream valueStream = analyzer.tokenStream(new StringReader(fieldValue),
false,true);
while ((token = valueStream.next()) != null) {
int startOffset = token.startOffset();
int endOffset = token.endOffset();
//Do some calculation based on startOffset & endOffset
}
============================================================================
//Lucene 8.11.2
========================================================================
TokenStream valueStream = analyzer.tokenStream(field, new
StringReader(fieldValue));
CharTermAttribute charValueTermAttribute =
valueStream.addAttribute(CharTermAttribute.class);
while (valueStream.incrementToken()) {
String termValueText = charValueTermAttribute.toString();
//How to get startOffset & endOffset as like in Lucene 2.4
//Do some calculation based on startOffset & endOffset
}
Please let me know, if there is any further information is required from my
side.
Regards
Rajib
