DONE. > -----Original Message----- > From: Michael McCandless [mailto:luc...@mikemccandless.com] > Sent: Monday, July 13, 2009 11:53 AM > To: java-user@lucene.apache.org > Subject: Re: OOM with 2.9 > > Ahh good point. I agree it makes sense to make MMapDir's chunking > user-controllable. Can you open an issue? > > Mike > > On Mon, Jul 13, 2009 at 5:49 AM, Uwe Schindler<u...@thetaphi.de> wrote: > >> On Sun, Jul 12, 2009 at 10:51 AM, eks dev<eks...@yahoo.co.uk> wrote: > >> > >> > MMapDirectory has support for chunking (Ineteger.MAX_VALUE) anyhow... > >> maybe for such cases this threshold can become user settable. I will > try > >> to experiment with it (I am talking about MMapDirectory -> private > final > >> int MAX_BBUF = Integer.MAX_VALUE;) > >> > >> This chunking doesn't reduce virtual memory usage, because the buffers > >> are all mapped up front on opening the IndexInput. > >> > >> It's only done because of a limitation of ByteBuffer (it uses int not > >> long to address the values). > > > > The idea of chunking is because of address space fragmentation. If the > Java > > VM allocates or frees memory (e.g. when expanding the heap from Xms to > Xmx), > > the address space of the process gets fragmented. If the index is very > > large, the byte buffer must fit in complete as one fragment into the > address > > space, which may easily fail when you only have 32 bit. > > > > If the file is mapped in different chunks, each chunk can be somewhere > else > > in address space (e.g., not only one 1 Gig junk is needed, there can be > ten > > 100 Meg junks at different places). > > > > Uwe > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > > For additional commands, e-mail: java-user-h...@lucene.apache.org > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org
--------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
