On Thu, Nov 8, 2012 at 11:30 AM, Robert Muir <[email protected]> wrote: > Why are you retrieving thousands of stored fields?
really you should roll your own codec for this and specialize. This is a very unique usecase. simon > > I don't think we should add such an API: stored fields shouldn't be > used like that, only for things like summary results, so the > possibility of two documents being in the same block is not so high > anyway. > > And i think the api to stay the way it is (simple visitor) to > encourage the fact that people shouldnt use stored fields for > "processing". > > On Thu, Nov 8, 2012 at 2:56 AM, eksdev <[email protected]> wrote: >> Just a theoretical question, would it make sense to add some sort of >> StoredDocument[] bulkGet(int[] docId) to fetch multiple stored documents in >> one go? >> >> The reasoning behind is that now with compressed blocks random-access gets >> more expensive, and in some cases a user needs to fetch more documents in >> one go. If it happens that more documents come from one block it is a win. I >> would also assume, even without compression , bulk access on sorted docIds >> cold be a win (sequential access)? >> >> Does that make sense, is it doable? Or even worse, does it already exist :) >> >> By the way, I am impressed how well compression does, even on really short >> stored documents, approx. 150b we observe 35% reduction. Fetching 1000 >> short documents on fully cached index is observably slower (2-3 times), but >> as soon as you memory gets low, compression wins quickly. Did not test it >> thoroughly, but looks good so far. Great job! >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
