Hello, Yes, that is exactly what MMapDirectory.setPreload is trying to do, but not promises (it is best effort). I think it asks the OS to touch all pages in the mapped region so they are cached in RAM, if you have enough RAM.
Make your JVM heap as low as possible to let the OS have more RAM to use to load your index. Mike McCandless http://blog.mikemccandless.com On Sun, Dec 13, 2020 at 4:18 PM <baris.ka...@oracle.com> wrote: > Hi,- > > it would be nice to create a Lucene index in files and then effectively > load it into memory once (since i use in read-only mode). I am looking into > if this is doable in Lucene. > > i wish there were an option to load whole Lucene index into memory: > > Both of below urls have links to the blog url where i quoted a very nice > section: > > > https://lucene.apache.org/core/8_5_0/core/org/apache/lucene/store/MMapDirectory.html > > https://lucene.apache.org/core/8_5_2/core/org/apache/lucene/store/MMapDirectory.html > > This following blog mentions about such option > to run in the memory: (see the underlined sentence below) > > > https://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html?m=1 > > MMapDirectory will not load the whole index into physical memory. Why > should it do this? We just ask the operating system to map the file into > address space for easy access, by no means we are requesting more. Java and > the O/S optionally provide the option to try loading the whole file into > RAM (if enough is available), but Lucene does not use that option (we may > add this possibility in a later version). > > My question is: is there such an option? > is the method setPreLoad for this purpose: > to load all Lucene lndex into memory? > > I would like to use MMapDirectory and set my > JVM heap to 16G or a bit less (since my index is > around this much). > > The Lucene 8.5.2 (8.5.0 as well) javadocs say: > public void setPreload(boolean preload) > Set to true to ask mapped pages to be loaded into physical memory on init. > The behavior is best-effort and operating system dependent. > > For example Lucene 4.0.0 does not have setPreLoad method. > > > https://lucene.apache.org/core/4_0_0/core/org/apache/lucene/store/MMapDirectory.html > > Happy Holidays > Best regards > > > Ps. i know there is also BytesBuffersDirectory class for in memory Lucene > but this requires creating Lucene Index on the fly. > > This is great for only such kind of Lucene indexes that can be created > quickly on the fly. > > Ekaterina has a nice article on this BytesBuffersDirectory class: > > > https://medium.com/@ekaterinamihailova/in-memory-search-and-autocomplete-with-lucene-8-5-f2df1bc71c36 > >
