Re: Document Decay

2009-12-05 Thread Grant Ingersoll

On Dec 4, 2009, at 1:56 AM, brad anderson wrote:

 Hi,
 
 I'm looking for a way to have the score of documents decay over time. I want
 older documents to have a lower score than newer documents.
 
 I noted the ReciprocalFloatFunction class. In an example it seemed to be
 doing just this when you set the function to be:
 
 recip(ms(NOW,mydatefield),3.16e-11,1,1)
 
 This is supposed to degrade the score to half its value if the mydatefield
 is 1 year older  than the current date. My question with this is, is it
 making the document score go down to 0.5 or is it making the document score
 1/2 of its original value.
 
 i.e.
 
 The document has score 0.8
 
 Will the score be 0.4 or 0.5 after using this function?


Actually, the value of the function gets added (it can be multiplied, too, with 
other params) to the score for the document.  You can see this by adding a 
debugQuery=true value to your request which allows you to examine the explains.

 
 Also, are there better alternatives to deal with document decay?

Some people like a different decay that does something like:  today is better 
than yesterday, yesterday is better than last week and last week is better than 
last month, etc. (in a non-linear way).  Todo this, you would need to implement 
your own function, I think.

--
Grant Ingersoll
http://www.lucidimagination.com/

Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids) using 
Solr/Lucene:
http://www.lucidimagination.com/search



Document Decay

2009-12-03 Thread brad anderson
Hi,

I'm looking for a way to have the score of documents decay over time. I want
older documents to have a lower score than newer documents.

I noted the ReciprocalFloatFunction class. In an example it seemed to be
doing just this when you set the function to be:

 recip(ms(NOW,mydatefield),3.16e-11,1,1)

This is supposed to degrade the score to half its value if the mydatefield
is 1 year older  than the current date. My question with this is, is it
making the document score go down to 0.5 or is it making the document score
1/2 of its original value.

i.e.

The document has score 0.8

Will the score be 0.4 or 0.5 after using this function?

Also, are there better alternatives to deal with document decay?

Thanks,
Stephen