It's already available in the latest 0.8-SNAPSHOT (there's no 0.8 release yet).
B. On 9 November 2011 17:43, Manokaran K <[email protected]> wrote: > On Wed, Nov 9, 2011 at 4:26 PM, Robert Newson <[email protected]>wrote: > >> Hi, >> >> This is probably because 0.8-SNAPSHOT uses a version of Lucene later >> than 3.1, where the behavior of the standard tokenizer. I've added >> another tokenizer option called "classic" which gives the same results >> as the pre-3.1 tokenizer. >> >> > Thanks very much. Is the classic tokenizer already available in 0.8 or are > you yet to push the changes to github? > > regds, > mano >
