Sorting based on calculations at search time

2004-12-10 Thread Gurukeerthi Gurunathan
Hello,
 
 I'd like some suggestions on the following scenario. 
 Say I have an index with a stored, indexed field called
'weight'(essentially an int stored as string). I'd like to sort in
descending order of final weight, the search results by performing a
calculation involving the lucene score for each hits. For our
discussion, the calculation can be as simple as multiplying the lucene
score with the value from the field 'weight' to get final weight. The
search results can run into thousands of documents. Though finally I may
need only the top X number of documents, I wouldn't know what the top X
would be until I perform this calculation and sort it.
 The obvious way is to do a post processing of the hits iterator,
storing it in memory, performing this calculation and sorting it. Is
there any other better solution for this?
 
Thanks,
Guru.
 
 
*
Gurukeerthi Gurunathan
Third Pillar Systems
San Mateo, CA
650-372-1200x229
 


RE: Sorting based on calculations at search time

2004-12-10 Thread Gurukeerthi Gurunathan
Thanks Otis for your response and compliments (wish I was a lucene guru
like you guys :-)

I believe you are talking about the boost factor for fields or documents
while searching. That does not apply in my case - maybe I am missing a
point here. 
The weight field I was talking about is only for the calculation
purpose, I am not searching on that field (it can be just a stored,
unindexed field). The main searching happens on other fields(like title,
keywords etc.) for which I am already using some boost factor. The
problem starts after I search and got some set of results - all I want
here is the result to be ordered by a number that is a multiplication of
lucene score and the weight field value for each document. 

I understand that without iterating thru the hits I cannot retrieve the
score and the weight for each document - which is why I'd like this
calculation and ordering to happen while searching so that I can avoid
the iteration over the entire hits. If it involves working on the lucene
source code, please point me to the right class or package that I should
be dealing with.

Thanks again,
Guru.

-Original Message-
From: Otis Gospodnetic [mailto:[EMAIL PROTECTED] 
Sent: Friday, December 10, 2004 3:13 PM
To: Lucene Users List
Subject: Re: Sorting based on calculations at search time

Guru (I thought my first name was OK until now),

Have you tried using boosts for that?  You can boost individual Document
Fields when indexing, and/or you can boost individual Documents, thus
giving some more and some less 'weight', which will have an effect on
the final score.

Otis



--- Gurukeerthi Gurunathan [EMAIL PROTECTED] wrote:

 Hello,
  
  I'd like some suggestions on the following scenario. 
  Say I have an index with a stored, indexed field called 
 'weight'(essentially an int stored as string). I'd like to sort in 
 descending order of final weight, the search results by performing a 
 calculation involving the lucene score for each hits. For our 
 discussion, the calculation can be as simple as multiplying the lucene

 score with the value from the field 'weight' to get final weight. The 
 search results can run into thousands of documents. Though finally I 
 may need only the top X number of documents, I wouldn't know what the 
 top X would be until I perform this calculation and sort it.
  The obvious way is to do a post processing of the hits iterator, 
 storing it in memory, performing this calculation and sorting it. Is 
 there any other better solution for this?
  
 Thanks,
 Guru.
  
  
 *
 Gurukeerthi Gurunathan
 Third Pillar Systems
 San Mateo, CA
 650-372-1200x229
  
 


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



RE: Sorting based on calculations at search time

2004-12-13 Thread Gurukeerthi Gurunathan
Ah! It makes sense now... Thanks for the clarification Hoss. I think
it'll work in my case as I need to perform this calculation for every
search...

-Guru. 

-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Chris Hostetter
Sent: Friday, December 10, 2004 10:21 PM
To: Lucene Users List
Subject: RE: Sorting based on calculations at search time

: I believe you are talking about the boost factor for fields or
documents
: while searching. That does not apply in my case - maybe I am missing a
: point here.
: The weight field I was talking about is only for the calculation

Otis is suggesting that you set the boost of the document to be your
weight value.  That way Lucene will automaticly do your multiplucation
calculation when determining the score

The down side of this, is that i don't think there's anyway to keep it
from influencing the score on every search, so it's not something you
could use only on some queries.


-Hoss


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]