On 22-Jan-08, at 4:20 PM, Phillip Farber wrote:
We would need all 7M ids scored so we could push them through a filter query to reduce them to a much smaller number on the order of 100-10,000 representing just those that correspond to items in a collection.
You could pass the filter to Solr to improve the speed dramatically.
So to ask again, do you think it's possible to do this in, say, under 15 seconds? (I think I'm giving up on 0.5 sec. ...)
At this point, no-one is going to be able to answer you question unless they have done something similar. The largest individual index I've worked with is on the order of 10GB, and one thing I've learned is to not extrapolate several orders of magnitude beyond my experience.
-Mike
