Thanks Walter, My requirements are this:
1. Query the index for entries matching keyword. 2. remove any entries that are below a threshold score from the external system I'm looking at building a custom field type similar to ExternalFileField that can dole out a ValueSource that calls my external system. Jim Walter Underwood wrote: > > You might be able to split the ranking into a common score and > a dynamic score. Return the results nearly the right order, then > do a minimal reordering after. If you plan to move a result by > a maximum of five positions, then you could fetch 15 results to > show 10 results. That is far, far cheaper than fetching all > results and ranking them all. > > I wrote a description of the client-side part of this last year: > > http://wunderwood.org/most_casual_observer/2007/04/progressive_reranking.htm > l > > wunder > > On 7/29/08 5:59 PM, "Jim Murphy" <[EMAIL PROTECTED]> wrote: > >> >> If figured that it would be - but the rankings are dynamically >> calculated. >> I'd like to limit the number of calculations performed for this very >> reason. >> Still not sure if this approach will be better than naivly filtering docs >> after the query has happened. >> >> Reading about ValueSource thanks... >> >> Jim >> >> >> >> Yonik Seeley wrote: >>> >>> Calling out will be an order of magnitude (or two) slower compared to >>> moving the rankings into Solr, but it is doable. See ValueSource >>> (it's used by FunctionQuery). >>> >>> -Yonik >>> >>> On Tue, Jul 29, 2008 at 8:23 PM, Jim Murphy <[EMAIL PROTECTED]> >>> wrote: >>>> >>>> I take it I can add my own functions that would take care of calling >>>> out >>>> to >>>> my external ranking system? >>>> >>>> Looking for docs on that... >>>> >>>> Jim >>>> >>>> >>>> Yonik Seeley wrote: >>>>> >>>>> A function query might fit your needs... you could move some or all of >>>>> your external ranking system into Solr. >>>>> >>>>> -Yonik >>>>> >>>>> On Tue, Jul 29, 2008 at 7:08 PM, Jim Murphy <[EMAIL PROTECTED]> >>>>> wrote: >>>>>> >>>>>> I need to store 100 million documents in our Solr instance and be >>>>>> able >>>>>> to >>>>>> retrieve them with simple term queries - keyword matches. I'm NOT >>>>>> implementing a search application where documents are scored and >>>>>> ranked...they either match the keywords or not. Also, I have an >>>>>> external >>>>>> ranking system that I need to use to filter and order the search >>>>>> results. >>>>>> >>>>>> My requirements are for the very fast and reliable retrieval so I'm >>>>>> trying >>>>>> to figure a place to hook in or customize Solr/Lucene to just do the >>>>>> simplest thing, reliably and fast. >>>>>> >>>>>> 1. A naive approach would be to implement a handler, let the query >>>>>> happen >>>>>> normally then perform N lookups to my external scoring system then >>>>>> filter >>>>>> and sort the documents. It seems I may be doing a lot of extra work >>>>>> that >>>>>> way, especially with paging results and who knows what I'd doing to >>>>>> the >>>>>> cache. >>>>>> >>>>>> 2. Create a custom FieldType that is virtual and calls out to my >>>>>> external >>>>>> system? Then queries could be written to return all docs > my rank. >>>>>> >>>>>> 3. Implement custom Query, Weight, Scorer (et al) implementations to >>>>>> minimize the "Search Stuff" and just delegate calls to my external >>>>>> ranking >>>>>> system. >>>>>> >>>>>> 4. A filter of some kind? >>>>>> >>>>>> >>>>>> I'd love to get a sanity check on any of these approaches or some >>>>>> recommendations. >>>>>> >>>>>> Thanks >>>>>> >>>>>> Jim >>>>> >>>>> >>>> >>>> -- >>>> View this message in context: >>>> http://www.nabble.com/Using-Solr-for-Info-Retreval-not-so-much-Search...-tp1 >>>> 8723102p18723877.html >>>> Sent from the Solr - User mailing list archive at Nabble.com. >>>> >>>> >>> >>> > > > -- View this message in context: http://www.nabble.com/Using-Solr-for-Info-Retreval-not-so-much-Search...-tp18723102p18724853.html Sent from the Solr - User mailing list archive at Nabble.com.
