Hi Israel,

you can solve your problem at search time by passing a custom Similarity class that looks something like this:

  private Similarity similarity = new DefaultSimilarity() {
    public float tf(float v) {
      return 1f;
    }
    public float tf(int i) {
      return 1f;
    }
  };


See javadocs for details.

  karl

25 dec 2008 kl. 14.20 skrev Israel Tsadok:

A recurring problem I have with Lucene results is when a document contains the same word over and over again. If for some reason I have a document containing "badger badger badger badger badger badger badger badger", it
would appear high on the search results for "badger", even though it's
usually a useless document.
What I would like to do is ignore repeating words when counting the term
frequency. At first, I thought I could achieve this by indexing with a
TokenFilter that would skip repeated tokens, but then a search for e.g.
"Rochelle Rochelle" would return no results.

What I would like is to index all 8 "badger"s, but have the frequency of
"badger" saved as 1. Is that even possible?

Digging around in Lucene code, I found term frequency calculations
in FreqProxTermsWriterPerField.addTerm() - is that where I need to look?

Any help would be appreciated.
Israel


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org

Reply via email to