Hello Rajib,
Check OffsetAttribute.
On Tue, Jan 30, 2024 at 1:31 PM Saha, Rajib <rajib.s...@sap.com.invalid>
wrote:
> Hi,
>
> In our project for Lucene migration from 2.4.0 to 8.11.2, we need your
> suggestion to address a case.
> With Lucene 2.4.0, we were using the kind of below code snippet.
> With Lucene 8.11.2[Written snippet below], we need to extract the
> startOffset & endOffset value for further some calculation similar to
> Lucene 2.4.0.
> Is there any easy way/API to extract the values from tokenStream?
>
> //Lucene 2.4.0
> ===========================================================================
> Token token;
> TokenStream valueStream = analyzer.tokenStream(new
> StringReader(fieldValue), false,true);
> while ((token = valueStream.next()) != null) {
> int startOffset = token.startOffset();
> int endOffset = token.endOffset();
>
> //Do some calculation based on startOffset & endOffset
> }
>
> ============================================================================
>
>
> //Lucene 8.11.2
> ========================================================================
> TokenStream valueStream = analyzer.tokenStream(field, new
> StringReader(fieldValue));
> CharTermAttribute charValueTermAttribute =
> valueStream.addAttribute(CharTermAttribute.class);
> while (valueStream.incrementToken()) {
> String termValueText = charValueTermAttribute.toString();
>
> //How to get startOffset & endOffset as like in Lucene 2.4
>
> //Do some calculation based on startOffset & endOffset
> }
>
> Please let me know, if there is any further information is required from
> my side.
>
> Regards
> Rajib
>
--
Sincerely yours
Mikhail Khludnev
