I am doing a search using Lucene, and when I get the search results (hits), I want to be able to get the number of tokens in a certain field.
Is this possible? Where is this sort of information stored? I know the IndexSearcher.Explain can get some information, but it seems mostly in free text and I cannot see the numTokens value. Ideally what I would like to be able to do is get the NumTokens value and the amount of hits within a field and display to the user what the "keyword density" is as a percentage. Any pointers would be greatly appreciated. Thanks, Andrew
