Why are you retrieving thousands of stored fields? I don't think we should add such an API: stored fields shouldn't be used like that, only for things like summary results, so the possibility of two documents being in the same block is not so high anyway.
And i think the api to stay the way it is (simple visitor) to encourage the fact that people shouldnt use stored fields for "processing". On Thu, Nov 8, 2012 at 2:56 AM, eksdev <[email protected]> wrote: > Just a theoretical question, would it make sense to add some sort of > StoredDocument[] bulkGet(int[] docId) to fetch multiple stored documents in > one go? > > The reasoning behind is that now with compressed blocks random-access gets > more expensive, and in some cases a user needs to fetch more documents in > one go. If it happens that more documents come from one block it is a win. I > would also assume, even without compression , bulk access on sorted docIds > cold be a win (sequential access)? > > Does that make sense, is it doable? Or even worse, does it already exist :) > > By the way, I am impressed how well compression does, even on really short > stored documents, approx. 150b we observe 35% reduction. Fetching 1000 short > documents on fully cached index is observably slower (2-3 times), but as > soon as you memory gets low, compression wins quickly. Did not test it > thoroughly, but looks good so far. Great job! > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
