On Dec 19, 2007 10:59 AM, Rishabh Joshi <[EMAIL PROTECTED]> wrote: > I have created my own Tokenizer and I am indexing the documents using the > same. > > I wanted to know if there is a way to retrieve the tokens (created by my > custom tokenizer) from the index.
If you want the tokens in the index, see the luke request handler. If you want the tokens for a specific document, it's more complicated... Lucene maintains an *inverted* index... terms point to documents, so by default there is no way to ask for all of the terms in a certain document. One could ask lucene to store the terms for certain fields (called term vectors), but that requires extra space in the index, and solr doesn't yet have a way to ask that they be retrieved. What are you trying to do with the tokens? -Yonik
