Hi All -

What is the best way to load a RAM Directory from a FS Directory, and 
periodically reload the RAM Directory to pick up new documents?

The scenario I have is I create several large directories which I create to a 
file system, then load them into ram for faster searching.
They takes several hours to create, so i want to retain a file copy so in the 
event of a service/server crash or reboot, they can load again
in a few seconds.  Once a day I append or recreate the FS Directories with new 
entries, and then reload the RAM Directories to pick up the
new entries.

Problem is, what I am doing seems to cause a memory leak.  The directories take 
~12 GB of RAM to load, but bloats
to 24 GB (the -Xmx setting) after a few reloads and stays there.

        // Map to hold single instance of the IndexSearcher, 1 per RAM 
Directory, to be reused across requests.
        private static final Map<IndexName, IndexSearcher> searchers = new 
HashMap<IndexName, IndexSearcher>();
        
        // Creates the IndexSearcher and stores it in the static map.
        public static void load(IndexName index, ...) {
                // sync... etc
                RAMDirectory dir = new 
RAMDirectory(FSDirectory.getDirectory(directoryPath);
                IndexReader reader = IndexReader.open(dir, true);
                IndexSearcher searcher = new IndexSearcher(reader);
                searchers.put(index, searcher);
        }
        
        public static IndexSearcher get(IndexName index, ...) {
                // sync... etc
                return searchers.get(index);
        }
        
        // After an indexing service appends the existing FS Directory, this 
reloads it.
        public static void reload(IndexName index, ...) {
                // sync... etc
                IndexSearcher searcher = searchers.get(index);
                FSDirectory fsDirectory = 
FSDirectory.getDirectory(directoryPath);
                Directory ramDirectory = searcher.getIndexReader().directory();
                Directory.copy(fsDirectory, ramDirectory, false);
        }

I've tries some theme and variations on this with the same issue.

TIA!

gd


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org

Reply via email to