Mark,
Question: how much memory I need for 25,000,000 docs if I do a sort by
<string> field, 256 bytes. 6.4Gb?
Quoting Mark Miller <[EMAIL PROTECTED]>:
Because to sort efficiently, Solr loads the term to sort on for each
doc in the index into an array. For ints,longs, etc its just an array
the size of the number of docs in your index (i believe deleted or
not). For a String its an array to hold each unique string and an array
of ints indexing into the String array.
So if you do a sort, and search for something that only gets 1 doc as a
hit...your still loading up that field cache for every single doc in
your index on the first search. With solr, this happens in the
background as it warms up the searcher. The end story is, you need more
RAM to accommodate the sort most likely...have you upped your xmx
setting? I think you can roughly say a 2 million doc index would need
40-50 MB (depending and rough, but to give an idea) per field your
sorting on.
- Mark