[ https://issues.apache.org/jira/browse/LUCENE-8564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16691605#comment-16691605 ]
Alan Woodward commented on LUCENE-8564: --------------------------------------- I reworked this slightly; we now have GraphTokenFilter as an abstract extension of TokenFilter, rather than using a separate standalone object. This makes reset()/end() easier to handle. > Make it easier to iterate over graphs in tokenstreams > ----------------------------------------------------- > > Key: LUCENE-8564 > URL: https://issues.apache.org/jira/browse/LUCENE-8564 > Project: Lucene - Core > Issue Type: Task > Reporter: Alan Woodward > Assignee: Alan Woodward > Priority: Major > Attachments: LUCENE-8564.patch, LUCENE-8564.patch > > > We have a number of TokenFilters that read ahead in the token stream (eg > synonyms, shingles) and ideally these would understand token graphs as well > as linear streams. FixedShingleFilter already has some mechanisms to deal > with graphs; this issue is to extract this logic into a GraphTokenStream > class that can then be reused by other token filters -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org