bq: Does the new compressed stored field format in Solr 4.1 do anything to reduce the number of disk seeks required to retrieve all document fields?
Probably, but I doubt by a whole lot. Although I confess I really don't know the guts. Let's assume that all the stored content for a doc is contiguous. The odds of having more than one doc in a block read from disk goes up with compression which would reduce the number of seeks. But, the odds of any of the top 20 docs in a corpus of, say, 20M docs being close enough for this to happen is probably pretty small. But read Uwe's excellent blog on MMapDirectory here: http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html Best, Erick On Tue, Feb 19, 2013 at 11:24 PM, Shawn Heisey <s...@elyograg.org> wrote: > On 2/19/2013 6:47 PM, Erick Erickson wrote: > >> It Depends (tm). Storing data in a Solr index pretty much just consumes >> disk space, the *.fdt and *.fdx files aren't really germane to the amount >> of memory needed for search. There will be some additional memory >> requirements for the document cache though. And you'll also have resources >> consumed if you use the &fl=*, there's more disk seeking going on to fetch >> the fields. >> > > Does the new compressed stored field format in Solr 4.1 do anything to > reduce the number of disk seeks required to retrieve all document fields? > > Thanks, > Shawn > >
