Classic is ... "classic" ... it exists largely for historical purposes to provide a tokenizer that does exactly what the javadocs say it does (regarding punctuation, "produc numbers", and email addresses), so that people who depend on that behavior can continue to rely on it.
Standard is ... "standard" ... it implements that Unicode Standard text segmentation rules. : Date: Fri, 20 Oct 2017 18:58:35 +0530 : From: Chitra <[email protected]> : Reply-To: [email protected] : To: Lucene Users <[email protected]> : Subject: Re: ClassicAnalyzer Behavior on accent character : : Hi, : I found the difference and understand the behavior of both : tokenizers appropriately. : : Could you please suggest me which one is the better to use : ClassicTokenizer/StandardTokenizer? : : -- : Regards, : Chitra : -Hoss http://www.lucidworks.com/ --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
