On 12/17/13 1:16 PM, Chris Hostetter wrote:
As i mentioned in the blog above, as long as you have a uniqueKey field that supports range queries, bulk exporting of all documents is fairly trivial by sorting on your uniqueKey field and using an fq that also filters on your uniqueKey field modify the fq each time to change the lower bound to match the highest ID you got on the previous "page".
Aha, very nice suggestion, I hadn't thought of this, when myself trying to figure out decent ways to 'fetch all documents matching a query' for some bulk offline processing.
One question that I was never sure about when trying to do things like this -- is this going to end up blowing the query and/or document caches if used on a live Solr? By filling up those caches with the results of the 'bulk' export? If so, is there any way to avoid that? Or does it probably not really matter?
Jonathan
