In Lucene 4.5 (coming out any day now) we've switched by default to a "mostly on disk" impl for doc values.
Before that, you can use DiskDocValuesFormat instead. But you'll need to re-index (or create a new index and use IW.addIndexes) to cutover your current index to the DiskDVFormat. Mike McCandless http://blog.mikemccandless.com On Wed, Oct 2, 2013 at 2:11 PM, Steven Schlansker <ste...@likeness.com> wrote: > Hi, > > I have a search application using Lucene 4.4.0 with various BinaryDocValues > and SortedSetDocValues. > We use MMapDirectory to help keep the Java heap small / GC pause times short > and instead rely on the OS buffer cache to keep things fast, which I gather > is generally considered a "best practice" around here. > As our index grows, I've noticed that we are getting GC pauses and later OOM > errors when reloading a new index due to gigabytes of byte[][]s held by > Lucene42DocValuesProducer, specifically the PagedBytes.Reader.blocks from > within Lucene42DocValuesProducer.loadBinary > > I would have expected DocValues fields to use mapped bytes instead of copying > into the Java heap much as the "main" index data is. Is this a technical > limitation, a "we haven't gotten there yet" feature request, or something > different entirely? > > Thanks for helping my understanding, > Steven > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
