Re: Custom Sorting Algorithm
Hi, You can use one of the exiting function queries (if they fit your need) or write a custom function query to reorder the results of a query. Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message From: wojtekpia wojte...@hotmail.com To: solr-user@lucene.apache.org Sent: Wednesday, February 4, 2009 2:28:56 PM Subject: Custom Sorting Algorithm Is an easy way to choose/create an alternate sorting algorithm? I'm frequently dealing with large result sets (a few million results) and I might be able to benefit domain knowledge in my sort. -- View this message in context: http://www.nabble.com/Custom-Sorting-Algorithm-tp21837721p21837721.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Custom Sorting Algorithm
That's not quite what I meant. I'm not looking for a custom comparator, I'm looking for a custom sorting algorithm. Is there a way to use quick sort or merge sort or... rather than the current algorithm? Also, what is the current algorithm? Otis Gospodnetic wrote: You can use one of the exiting function queries (if they fit your need) or write a custom function query to reorder the results of a query. -- View this message in context: http://www.nabble.com/Custom-Sorting-Algorithm-tp21837721p21838804.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Custom Sorting Algorithm
It would not be simple to use a new algorithm. The current implementation takes place at the Lucene level and uses a priority queue. When you ask for the top n results, a priority queue of size n is filled with all of the matching documents. The ordering in the priority queue is the sort. The on Sort method orders by relevance score - the Sort method orders by field, relevance, or doc id. - Mark wojtekpia wrote: That's not quite what I meant. I'm not looking for a custom comparator, I'm looking for a custom sorting algorithm. Is there a way to use quick sort or merge sort or... rather than the current algorithm? Also, what is the current algorithm? Otis Gospodnetic wrote: You can use one of the exiting function queries (if they fit your need) or write a custom function query to reorder the results of a query.
Re: Custom Sorting Algorithm
Ok, so maybe a better question is: should I bother trying to change the sorting algorithm? I'm concerned that with large data sets, sorting becomes a severe bottleneck (this is an assumption, I haven't profiled anything to verify). Does it become a severe bottleneck? Do you know if alternate sort algorithms have been tried during Lucene development? markrmiller wrote: It would not be simple to use a new algorithm. The current implementation takes place at the Lucene level and uses a priority queue. When you ask for the top n results, a priority queue of size n is filled with all of the matching documents. The ordering in the priority queue is the sort. The on Sort method orders by relevance score - the Sort method orders by field, relevance, or doc id. -- View this message in context: http://www.nabble.com/Custom-Sorting-Algorithm-tp21837721p21840299.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Custom Sorting Algorithm
On Wed, Feb 4, 2009 at 4:45 PM, wojtekpia wojte...@hotmail.com wrote: Ok, so maybe a better question is: should I bother trying to change the sorting algorithm? I'm concerned that with large data sets, sorting becomes a severe bottleneck (this is an assumption, I haven't profiled anything to verify). No... Lucene/Solr never sorts the complete result set. If you ask for the top 10 results, a priority queue (heap) of the current top 10 results is maintained... far more efficient and scalable than sorting all the hits at the end. -Yonik
