Re: Shingles Filter problems

2011-10-11 Thread Peyman Faratin
Hi Ian i think i found out the problem (from tests here http://www.devdaily.com/java/jwarehouse/lucene/contrib/analyzers/common/src/test/org/apache/lucene/analysis/shingle/ShingleAnalyzerWrapperTest.java.shtml) if you generate the query as a BooleanQuery then it seems to work. The following wor

Re: Shingles Filter problems

2011-10-11 Thread Ian Lea
Something does appear dodgy here. Using 3.4.0 the following very simple code, with no custom classes ShingleAnalyzerWrapper saw = new ShingleAnalyzerWrapper(LUCENE_34); QueryParser qp = new QueryParser(LUCENE_34, "t", saw); String s = "simple sentences rule"; Query

Shingles Filter problems

2011-10-11 Thread Peyman Faratin
Hi I have the following shinglefilter (Lucene 3.2) public TokenStream tokenStream(String fieldName, Reader reader) { StandardTokenizer first = new StandardTokenizer(Version.LUCENE_32, reader); StandardFilter second = new StandardFilter(Version.LUCEN