If you want the index to be stored completely in RAM, there is the ByteBuffer directory [1]. Though I do not see the point in putting an index in RAM, it will be cached in RAM regardless in the OS system IO cache.
1. https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/apache/lucene/store/bytebuffer/ByteBufferDirectory.java On Mon, Jun 4, 2012 at 10:55 AM, Cheng <zhoucheng2...@gmail.com> wrote: > My indexes are 500MB+. So it seems like that RAMDirectory is not good for > that big a size. > > My challenge, on the other side, is that I need to update the indexes very > frequently. So, do you think MMapDirectory is the solution? > > Thanks. > > On Mon, Jun 4, 2012 at 10:30 PM, Jack Krupansky > <j...@basetechnology.com>wrote: > >> From the javadoc for RAMDirectory: >> >> "Warning: This class is not intended to work with huge indexes. Everything >> beyond several hundred megabytes will waste resources (GC cycles), because >> it uses an internal buffer size of 1024 bytes, producing millions of >> byte[1024] arrays. This class is optimized for small memory-resident >> indexes. It also has bad concurrency on multithreaded environments. >> >> It is recommended to materialize large indexes on disk and use >> MMapDirectory, which is a high-performance directory implementation working >> directly on the file system cache of the operating system, so copying data >> to Java heap space is not useful." >> >> -- Jack Krupansky >> >> -----Original Message----- From: Cheng >> Sent: Monday, June 04, 2012 10:08 AM >> To: java-user@lucene.apache.org >> Subject: RAMDirectory unexpectedly slows >> >> >> Hi, >> >> My apps need to read from and write to some big indexes frequently. So I >> use RAMDirectory instead of FSDirectory, and give JVM about 2GB memory >> size. >> >> I notice that the speed of reading and writing unexpectedly slows as the >> size of the indexes increases. Since the usage of RAM is less than 20%, I >> think by default the RAMDirectory doesn't take advantage of the memory I >> assigned to JVM. >> >> What are the steps to improve the reading and writing speed of >> RAMDirectory? >> >> Thanks! >> Jeff >> >> ------------------------------**------------------------------**--------- >> To unsubscribe, e-mail: >> java-user-unsubscribe@lucene.**apache.org<java-user-unsubscr...@lucene.apache.org> >> For additional commands, e-mail: >> java-user-help@lucene.apache.**org<java-user-h...@lucene.apache.org> >> >> --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
