Dear Wiki user, You have subscribed to a wiki page or wiki category on "Solr Wiki" for change notification.
The following page has been changed by HossMan: http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters ------------------------------------------------------------------------------ = Analyzers, Tokenizers, and Token Filters = + == Overview == + When a document is indexed, its individual fields are subject to the analyzing and tokenizing filters that can transform and normalize the data in the fields. For example — removing blank spaces, removing html code, stemming, removing a particular character and replacing it with another. At indexing time as well as at query time you may need to do some of the above or similiar operations. For example, you might perform a [http://en.wikipedia.org/wiki/Soundex Soundex] transformation (a type of phonic hashing) on a string to enable a search based upon the word and upon its 'sound-alikes'. + + The lists below provide an overview of some of the more heavily used Tokenizers and !TokenFilters provided by Solr "out of the box" along with tips/examples of using them. This list should by no means be considered the "complete" list of all Analysis classes available in Solr, in addition to new classes being added on an ongoing basis, you can load your own custom Analysis code as a [SolrPlugins Plugin]. + + For a more complete list of what Tokenizes and TokenFilters come out of the box, please consult the [http://lucene.apache.org/solr/api/org/apache/solr/analysis/package-summary.html javadocs] for the analysis package. if you have any tips/tricks you'd like to mention about using any of these classes, please add them below. '''Note:''' For a good background on Lucene Analysis, it's recommended that you read the following sections in [http://lucenebook.com/ Lucene In Action]: @@ -11, +17 @@ Try searches for "analyzer", "token", and "stemming". - Here is a up-to-date and complete [http://svn.apache.org/viewvc/lucene/solr/trunk/src/java/org/apache/solr/analysis/ list] of filters/tokenizers included with Solr. [[TableOfContents]] - == Stemming ==
