Hi hariram, There may be other problems, but at a minimum you have two different analysis classes here. You’re printing the output stream from one (CustomSynynymAnalyzer, the source of which is not shown in your email), but constructing a query from a different one (CustomAnalyzer).
-- Steve www.lucidworks.com > On Jul 24, 2017, at 10:53 AM, hariram ravichandran > <hariramravichan...@gmail.com> wrote: > > I'm using Lucene 4.10.4 and trying to construct (shingles) combinations of > tokens. > > > Code: > > public class CustomAnalyzer extends Analyzer { > @Override > protected Analyzer.TokenStreamComponents createComponents(final String > fieldName, final Reader reader) { > final WhitespaceTokenizer src = new > WhitespaceTokenizer(getVersion(), reader); > TokenStream tok = new ShingleFilter(src, 2, 3); > tok = new ClassicFilter(tok); > tok = new LowerCaseFilter(tok); > // tok = new > SynonymFilter(tok,SynonymDictionary.getSynonymMap(),true); > return new Analyzer.TokenStreamComponents(src, tok); > } > } > > public class Test { > public static void main(String[] args) throws Exception { > CustomSynonymAnalyzer analyzer = new CustomSynonymAnalyzer(); > String queryStr = "cup board"; > TokenStream ts = new CustomAnalyzer().tokenStream("n", new > StringReader(queryStr)); > ts.reset(); > System.out.println("Tokens are :"); > while (ts.incrementToken()) { > System.out.print(ts.getAttribute(CharTermAttribute.class) + ", > "); > } > QueryParser parser = new QueryParser("n", analyzer); > Query query = null; > query = parser.parse(queryStr); > System.out.println("\nQuery is"); > System.out.print(query.toString()); > } > } > > > >> Output: >> Tokens are : >> cup, cup board, board >> Query is n >> n:cup n:board >> > > Tokens are printed as expected. And expecting the resulting query to be *n:cup > n:board n:cup board*. But tokens formed by shingle filter are not appended > to the query. I get only *n:cup n:board.* Where is my mistake? > > Thanks. --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org