Hi folks,
If I understand things correctly, you can use paging & sorting in a
SolrCloud environment. However, if I request the first 10 documents, a
distributed query will be launched to all shards requesting the top 10,
and then (Shards * 10) documents will then be sorted so that only the
top 10 is returned.
This is fine.
But I'm a little worried when going beyond the first page ... This
becomes (Page * shards * 10). I'm worried that in a 50 billion document
setup paging will just explode.
Does anyone have any experience with paging on large cloud setups?
Positive or negative? Or can anyone offer some reassurances or words of
caution with this approach?
Or should I tell my users that they can never go beyond Page X (which is
fine if the alternative is hell fire and brimstone).
Thanks,
- Bram
- SolrCloud & Paging on large indexes Bram Van Dam
-